OpenStack SIG Proposal -draft-

ExtremeTesting SIG

Status: Proposed (Proposal: http://lists.openstack.org/pipermail/openstack-sigs/....) will fill in as we send out proposal for the first time

Description: ExtremeTesting will integrate various projects to improve the capabiliy and practice of testing openstack in failure scenarios.  The ultimate goal is to add to the QA community to improve Robustness/Resiliency of OpenStack at the CI/CD gates.  The SIG will engage both developer and operators to widen the scope of failures and components exercised and to exercise the tests on their chosen openstack configuration submitting feedback back through QA pipeline to projects. 

Extreme testing  started as a development proposal which can be seen at: http://specs.openstack.org/openstack/openstack-user-stories/user-stories/proposed/openstack_extreme_testing.html.  Much of the early scoping and consideration of scope and project impact considerations can be seen in Extreme Testing-Vision+Arch.  Gap analysis

Potential Goals

• Provide the forum in which interested Operators, Performance and Reliability Engineers and Developers can collaborate on ways of verifying openstack reliability.
• Sharing in the specification and development of the methods/scripts for both driving workloads and creating outages for which the cloud and it's workload should persist
• Sharing in the job of running and reporting some subset of the functionality as part of the regular upstream QA process

Audience

• Operators of clouds and cloud based applications needing to prove out and address reliability
• Developers of QA/Tools (especially those interested in performance and reliability

SIG Leads

• Sampath Priyankara - samP
• Gautam Divgi -
• Sundar Krishnamoorthy (Deactivated)
• Jamey McCabe - jamemcc
  
  

Community Infrastructure

• Wiki: Eris - Extreme Testing Framework for OpenStack
• Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-sigs; [ExtremeTesting] for the tag
• IRC:  #openstack-qa
• Meetings:  Weekly: Agenda / details to be linked on wiki page

Alternative/Related efforts/Other notes

1. This SIG and it's output appear to be similar to a QA project.  it's possible to just exist as a QA subproject, but if so but then would seem to miss out on the promise of SIGs to bring in more collaborators from many directions.
   1. The QA project itself has been suggested to be converted to a SIG, that discussion elsewhere. If so this could certainly be adopted as objectives/initiatives of that.
2. Self Healing SIG - Assuming self healing is to
3. Fault Genes working group
4. The initial scope concentartes on Ansible driving functionality so seems to have some overlap to: https://wiki.openstack.org/wiki/Ansible_SIG
5. Many have and want to use the project Rally for this general use case: https://wiki.openstack.org/wiki/Rally. Rally is certainly envisioned to be the load generating component of extreme testing so is hoped to be one of the projects that collaborates in this SIG.  This effort seems to be a different one than Rally's original scope though so worth persuing a cross project effort.