Eucalyptus

Eutester Basics Part I

Introduction

In my time working for Eucalyptus, I have begun the Eutester project that aims to provide the framework for cloud architects and administrators to validate and benchmark their various AWS-compatible cloud infrastructure options.  Enterprises and research institutions are rushing to roll out their strategies for moving their mission critical applications and services to on-premise, public, or hybrid clouds. The end result of this strategy should be a fault tolerant and reliable compute, storage, and networking infrastructure. Unfortunately, choosing a private or hybrid IaaS solution to implement is not the end of the road for a cloud strategy. The next level of planning involves choosing and validating the interconnected components of the cloud infrastructure stack. The considerations necessary include, but are not limited to, choosing network topologies, hypervisors, component topology, and storage devices. What many  will end up with is a few different ways to implement the end goal, a scalable and robust self service infrastructure. In order to make “apples to apples” comparisons of these options, it is necessary to have a test library that is agnostic to the underlying components and treats each cloud implementation equally by presenting results in a unified and comparable way.

Eutester Basics

Eutester is written in Python and leverages the boto and paramiko libraries to provide the necessary tools for quickly implementing an automated test strategy. Boto is used in order to provide connections to EC2, S3, and IAM. Paramiko is leveraged to provide root access to nodes that are running the individual cloud components. The eutester module is separated into two main classes: eutester, which provides the necessary logic to get a test bootstrapped, and eucaops, which inherits from eutester and provides predefined routines that can validate operations against the cloud. Here is an example of the difference in using the eutester class by itself rather than using eucaops. Notice that when using both eutester and eucaops you have full access to the underlying EC2 and S3 connections provided by boto.

>>> from eutester import Eutester
>>> from eucaops import Eucaops
>>> eutester = Eutester(credpath="../credentials")
>>> eucaops = Eucaops(credpath="../credentials")
>>> eutester.ec2.get_all_images()
[Image:eri-168B3564, Image:eki-E9A03638, Image:emi-5D824306, Image:emi-30823A1C]
>>> eucaops.get_emi()
Image:emi-5D824306
>>> eucaops.ec2.get_all_images()
[Image:eri-168B3564, Image:eki-E9A03638, Image:emi-5D824306, Image:emi-30823A1C]

As you can see the only required parameter to get a test up and running is simply the path to a credentials directory that contains a eucarc file that can be used to extract both the access and secret keys as well as the hostname of the EC2 and S3 implementation. In the case of Eucalyptus these are the Cloud Controller and Walrus. Once the boto connections have been established the testing is ready to begin and you can start to send commands to the cloud:

>>> emi = eucaops.get_emi()
>>> reservation = eucaops.run_instance(emi)
[2012-03-04 17:46:42,205] [EUTESTER] [DEBUG]: Attempting to run image Image:emi-5D824306 in group default
[2012-03-04 17:46:42,976] [EUTESTER] [DEBUG]: Beginning poll loop for the 1 found in Reservation:r-F4F842A4
[2012-03-04 17:46:42,976] [EUTESTER] [DEBUG]: Beginning poll loop for instance Instance:i-87FB415E to go to running
[2012-03-04 17:48:16,953] [EUTESTER] [DEBUG]: Instance(i-87FB415E) State(running) Poll(9) time elapsed (93)
[2012-03-04 17:48:16,953] [EUTESTER] [DEBUG]: Instance:i-87FB415E is now in running
[2012-03-04 17:48:16,953] [EUTESTER] [DEBUG]: Instance i-87FB415E now in running state

The output here shows that the instance we ran successfully went to the running state, as expected. Once the instance is running it may be useful to ping the instance from the test machine to ensure we have connectivity.

>>> instance = reservation.instances[0]
>>> eucaops.ping(instance.public_dns_name, poll_count=1)
[2012-03-04 18:08:38,075] [EUTESTER] [DEBUG]: Attempting to ping 192.168.47.202
[2012-03-04 18:08:38,076] [EUTESTER] [DEBUG]: [root@localhost]# ping -c 1 192.168.47.202
[2012-03-04 18:08:48,081] [EUTESTER] [DEBUG]: PING 192.168.47.202 (192.168.47.202) 56(84) bytes of data.

— 192.168.47.202 ping statistics —
1 packets transmitted, 0 received, 100% packet loss, time 0ms

[2012-03-04 18:08:48,081] [EUTESTER] [CRITICAL]: Was unable to ping address
False

We were unable to ping the instance likely because the group that we launched it in, “default,” did not have the proper ports authorized. Lets go back and authorize the ports and retry the ping:

>>> eucaops.authorize_group_by_name(group_name="default", protocol="icmp",port="-1")
[2012-03-04 18:11:35,755] [EUTESTER] [DEBUG]: Attempting authorization of group
>>> eucaops.ping(instance.public_dns_name, poll_count=1)
[2012-03-04 18:12:18,445] [EUTESTER] [DEBUG]: Attempting to ping 192.168.47.202
[2012-03-04 18:12:18,445] [EUTESTER] [DEBUG]: [root@localhost]# ping -c 1 192.168.47.202
[2012-03-04 18:12:18,457] [EUTESTER] [DEBUG]: PING 192.168.47.202 (192.168.47.202) 56(84) bytes of data.
64 bytes from 192.168.47.202: icmp_seq=1 ttl=63 time=6.49 ms

— 192.168.47.202 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 6.498/6.498/6.498/0.000 ms

[2012-03-04 18:12:18,457] [EUTESTER] [DEBUG]: Was able to ping address
True

Now we can see that the instance is reachable. In these few steps we have validated the folllowing:

  • Hypervisor is properly operating and can run VMs
  • The image we used can boot and start networking properly
  • Opening ports on a security group is working properly
  • Networking from the CLC to CC and CC to Node controller is working as expected

These may seem like trivial checks but should not be taken for granted when first turning up an installation of a cloud platform.  Along with the checks of functionality we also received some benchmarks for this particular run instance request in which it shows that for the instance to go to running it took 93 seconds. If we rerun those few steps with various images we can get a feel for the performance of different images on the same cloud. Similarly if we use the same image against clouds running different hypervisors we can get a feel for the prep time each hypervisor requires to bring up an instance.

Here we have seen a case where things more or less worked as expected and were easy to debug and get running. When things are not quite working the way we expect, eutester allows you to start and stop monitoring the logs from all of your Eucalyptus components. This requires a slightly different initialization of the Eucaops or Eutester object wherein we include a root password for the Eucalyptus component nodes as well as a small configuration file that provides the topology of the components and thier hostnames. In my next blog post I’ll go over how Eutester can help you when your cloud is not functioning properly.

For an initial API doc please see:

https://github.com/eucalyptus/eutester/wiki/API-Documentation

Raise issues on the Github project page at:

https://github.com/eucalyptus/eutester/issues

All contributions are welcome, whether it be testing, refactoring code, new use cases, or added functionality.

Come hang out at #eucalyptus on freenode to discuss any topics related to Eutester or Eucalyptus in general.

About these ads
Standard

8 thoughts on “Eutester Basics Part I

  1. Pingback: Week 6 Getting eutester to work « sflynn1976

  2. Pingback: Researching EuTester « michaelkenny2

  3. Pingback: Installing and Configuring EuTester « michaelkenny2

  4. Ilpo says:

    Hi,

    Is there any way to run instances from eutester with user-data as a parameter?
    Would be nice to be able to verify euca user-data functionality with eutester also.
    Was looking around in the examples area and there was no reference to user-data.

    -Ilpo-

  5. Pingback: Work During Spring Break « michaelkenny2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s