Introduction to Aminator
The Netflix Cloud Platform has shown how a large scale system can be deployed in a public cloud and maintain an extreme level of performance and reliability. As Adrian Cockcroft has said in the past, Netflix focused on having functional and scalable code rather than worrying immediately about how to make it portable. At Eucalyptus we have been working over the past year on making sure that as many of the NetflixOSS projects that interact directly with the cloud can be used on Eucalyptus. This level of portability means that anyone with even 1 single linux box can use their system as a test bed for deploying NetflixOSS more broadly on a public cloud. So far at Eucalyptus we have working versions of Asgard, Simian Army, Aminator, and Edda. These tools are cornerstones for app deployment and monitoring with the NetflixOSS stack. In this post I will show how to use Aminator with your Eucalyptus cloud.
Aminator is a tool created by Netflix to populate and catalog application images that are the building blocks for any service infrastructure you can dream up. Aminator works by taking a “Foundation AMI” and mounting a snapshot from it in order to provision your application. It does this by mounting a volume, created from an image snapshot, to a running instance then performing a chroot that runs provisioners such as Chef or Ansible. Once the provisioning step is complete the volume is snapped and registered as an AMI. Aminator doesn’t stop there however. It also creates tags of the snapshot and the AMI so that they can be easily identified. Some of the information included in the tags is:
* Package version info
* Name of application
* Base AMI
Having this information allows an Aminator user to trace back the history of how one of their applications was deployed, also pinning the deployment to a particular person.
Some of the benefits of using Aminator for app deployment include:
- Ensuring exact and dependable recovery of previous application stack (including dependencies and software)
- Allows applications to be deployed in AutoScaling groups as each AMI is completely self contained version of the application
- Ensures application images are tagged with appropriate meta data for traceability
- Allows traceability of ownership of images (since Netflix uses one large AWS account)
With the addition of Eucalyptus cloud to deploy on you can enjoy the following:
- An internal test and development platform for NetflixOSS
- Gives application developers an easy way to catalog, build, and deploy their test applications
- Ensures a repeatable process is in place for creating an image that will eventually go into production
- Test changes to an image quickly/cheaply on local private infrastructure before deploying into production
Using Aminator in Eucalyptus
In order to run Aminator we will first need to build our Aminator instance (which will also be the AMI we use as the “Foundation AMI”).
- Download the Ubuntu Precise QCOW disk image to a machine that has the qemu-img tool
- Convert the QCOW image to RAW format using the following command:
- qemu-img convert -O raw ubuntu-12.04-server-cloudimg-amd64-disk1.img ubuntu-12.04-server-cloudimg-amd64-disk1.raw
- Once the image is converted start up an instance so we can create our “Foundation AMI”
- After the instance is booted copy the raw image to the instances ephemeral storage
- Attach a 2G volume to the instance
- Copy the disk file to the volume using dd:
- dd if=ubuntu-12.04-server-cloudimg-amd64-disk1.raw of=/dev/vdb
- Create a snapshot from your volume
- Register the snapshot as an image in your cloud. Remember this AMI ID as it will be what we pass to Aminator in later steps.
- Run an instance from the newly created image and log into it
In these steps we have now created the base image for future application deployments and created the instance where we will run our Aminator tasks. Next up we will install Aminator and the Eucalyptus plugin:
- Clone the Aminator repository
- git clone https://github.com/Netflix/aminator.git
- Edit the aminator/default_conf/environments.yml file and add the following block:
euca_apt_linux: cloud: euca distro: debian provisioner: apt volume: virtio blockdevice: virtio finalizer: tagging_ebs_euca
- Run the setup script twice from inside the aminator directory
- cd aminator;python setup.py install; python setup.py install
- Now clone the eucalyptus-cloud Aminator plugin and install it
- git clone https://github.com/aminator-plugins/eucalyptus-cloud; cd eucalyptus-cloud; python setup.py install
- Now that you have all the dependencies lets run an amination to install and label an Apache web server image:
- sudo aminate -e euca_apt_linux –ec2-endpoint <clc-ip> -B emi-CD544111 apache2
In the above command we have told Aminator a few things about what we want to do:
- -e: Use the Eucalyptus environment for provisioning a Linux machine with APT
- –ec2-endpoint: Use this IP to connect to the Eucalyptus cloud
- -B: Use this AMI (the one we registered in the steps above) as the base image
- apache2: Install the apache2 package and tag the appropriate version information onto the snapshot and image
After a few minutes Aminator should complete, letting you know which AMI it has registered for you.
Enjoy your new application deployment tool!