I have devoted my last 2 years to testing Eucalyptus. In that period the QA team and I have gone through many iterations of tools to find those that make us most efficient. It has become a never ending and enjoyable quest.
We have evolved our testing processes through the following stages:
- Using command line tools exclusively
- Writing scripts that call command line tools and parsing their output
- Writing scripts using a library to make test creation easier and more efficient without the need for command line tools
- Running scripts through a graphical tool in order to make test execution more flexible and simple
Each of these iterations was fueled by some tool chain that came along to solve a problem. My journey at Eucalyptus started between the second and third stages. Euca2ools were the go to favorite for manual testing and there was a library aptly named ec2ops floating around that wrapped euca2ools commands using Perl. Being that boto was backing euca2ools at the time I figured I would take a stab at creating a library calling boto directly that we could start to build our tests with, and thus Eutester was born entering us into the third phase. Once we had reached this point, we were able to quickly write and execute idempotent tests from the command line. After manual execution of our tests was passing consistently we were able to parametrize, run, and parallelize our tests using Jenkins. At this fourth phase we have taken a snapshot of our environment and are now able to share it with the rest of the community through our image catalog.
A few of the use cases Micro QA can help with are:
- Regression testing during development
- Functional testing after initial installation
- Load and stress testing before going into production
- Development platform for Eutester test cases
Benefits for users of Micro QA include:
- Known working automated tests
- Constantly increasing number of test scenarios
- Flexibility to add custom tests as needed for use cases which aren’t covered
How it works
The environment starts with an Ubuntu Precise Guest Image from Ubuntu’s cloud-images. Once downloaded, registered and started, I installed the Jenkins package. After Jenkins was up and running I installed a redirect for port 8080 to 80 so that users would not need to remember a port in order to access their Micro QA image and could simply hit: http://<instance-public-ip>. Once the Jenkins instance was reachable by the standard HTTP port (80) we began to add the dependencies for Eutester to the guest OS. The typical environment for Eutester scripts requires boto, paramiko, and virtualenv to isolate the script runtime environment. Once the Python dependencies were successfully installed into a virtualenv we then setup our projects in the Jenkins install. The jobs for Eutester and Eutester4j were then created with only a single required parameter, namely the contents of the eucarc file generated by the cloud. Each script checks out its own environment so that both development and stable Eutester versions can run side by side.
In order to install the Micro QA image follow the instructions here: https://github.com/eucalyptus/micro-qa/blob/master/README.md
- Once at the main page of the Micro QA instance, click the “Build” button on the right side of the Instance Suite project.
- Enter your eucarc file into the text box on the next screen
- Hit build at the bottom of the page
- You will be taken to the currently running job. Click the blue bar under the job timestamp on the left.
- Here you will see the console output for the test run.
- When the script completes, the bottom of the console output will display a summary of the results.