Testing Riak CS with Eucalyptus

Introduction

One of the beautiful things about working with IaaS is the disposable nature of instances. If they are not behaving properly due to a bug or have been misconfigured for some reason, instances can be terminated and rebuilt with more ease than debugging a long lived and churned through Linux system. As a quality engineer, this dispensability has become invaluable in testing and developing new tools without needing to baby physical or virtual machines.

One of the projects I have been working on lately is an easy deployment of Riak CS into the cloud in order to quickly and repeatedly test the object storage integration provided by Eucalyptus in the 4.0 release. Riak CS is a scalable and distributed object store that provides an S3 interface for managing objects and buckets.

Before testing the Eucalyptus orchestration of Riak CS (or any tool/backend/service that Euca supports for that matter), it is important to understand the basic activities that Eucalyptus will be performing on behalf of the user. Thankfully, Neil Soman wrote a great blog post about how our Riak CS integration is designed.

In this model  we can see that we require:

1. A multi-node Riak CS cluster
2. A load balancer
3. A machine to run the Eucalyptus Object Storage Gateway (OSG)

This topology is extremely simple to deploy in Eucalyptus 3.4 using our ELB and by using Vagrant to deploy our Riak CS cluster. Here’ s how to get your groove on.

Prerequisites

1. CentOS 6 image loaded into your cloud
2. Keypair imported or created in the cloud
3. Security group authorized for port 8080,8000 and 22
4. Install Vagrant

Deploy Riak CS

In order to deploy Riak CS in our cloud we will use Vagrant+Chef+Berkshelf as follows:

1. Install Vagrant plugins using the following commands:

   • vagrant plugin install vagrant-berkshelf
     vagrant plugin install vagrant-omnibus
     vagrant plugin install vagrant-aws

2. Import the dummy vagrant box necessary to use vagrant-aws:

   • vagrant box add centos dummy.box

3. Clone the following repository

   • git clone https://github.com/viglesiasce/vagrant-riak-cs-cluster.git

4. Edit the following items in the Vagrantfile to reflect the pre-requisites above and to point to your target cloud
   • aws.access_key_id
   • aws.secret_access_key
   • aws.keypair_name
   • aws.ami
   • override.ssh.private_key_path
   • aws.security_groups
   • aws.endpoint
5. Set the number of nodes to deploy at the top of the Vagrantfile:
   • https://github.com/viglesiasce/vagrant-riak-cs-cluster/blob/develop/Vagrantfile#L5
6.  Once the cloud options are set start the Vagrant “up” process which will deploy the Riak CS nodes and Stanchion:

   • RIAK_CS_CREATE_ADMIN_USER=1 vagrant up --provider=aws

7. Once Vagrant is complete, login to the first Riak CS node to get its private hostname:

   • vagrant ssh riak1 -c "curl http://169.254.169.254/latest/meta-data/local-hostname"

8. Join each node to the first that was deployed. For example, to join the second node to the cluster I would run:

   • vagrant ssh riak2 -c "riak-admin cluster join riak@<riak1-private-hostname>"
     vagrant ssh riak2 -c "riak-admin cluster plan; riak-admin cluster commit"

In order to get your access and secret keys login to http://riak1-public-ip:8000

Load Balance Your Riak CS Cluster

1. Create an ELB with the following command:

   • eulb-create-lb -z <AZ-of-your-riak-nodes> -l "lb-port=80, protocol=TCP, instance-port=8080,instance-protocol=TCP" RiakCS

2. The command above will return you the DNS name that you will use as the endpoint for the “objectstorage.s3provider.s3endpoint” property when setting up the OSG. From the sample output below we would use “RiakCS-229524229045.lb.home”

   • DNS_NAME        RiakCS-229524229045.lb.home

3. Register your Riak CS nodes with that load balancer:

   • eulb-register-instances-with-lb --instances <instance-id-1>,<instance-id-2> RiakCS

You have now successfully deployed a Riak CS cluster. You can stop here if you’d like but the real fun starts when you add IAM, ACL, versioning, multipart upload, and bucket lifecycle support to the mix using the Eucalyptus OSG.

True enthusiasts continue below.

Install and Configure the Eucalyptus OSG Tech Preview

1. Spin up another CentOS 6 instance in the same security group as used above
2. Follow the instructions found here to finish  the OSG installation and configuration, remember to use the DNS name returned in step 1 from above as the s3endpoint:
   • https://github.com/eucalyptus/eucalyptus/wiki/Scalable-Walrus-Tech-Preview-in-Eucalyptus-3.4