In order to run a successful private cloud in production it is necessary to monitor the host systems. Monitoring of the bare metal machines hosting your private cloud software will give you an idea of when and how you need to improve performance in order to continue to provide your users an optimal experience. Through my background in networking I was introduced to SNMP. Although not as simple as its name implies, toolchains and frameworks supply abstractions of the SNMP protocol that a system administrator can use to get a great feel of how and when their infrastructure is being utilized. In this tutorial I will be leveraging Zenoss Core for aggregation and display of SNMP data collected from my Eucalyptus hosts. The tutorial should get you up and running with monitoring of:
- Memory, disk, CPU, and network usage of bare metal machine as a whole
- Per process Memory, disk, CPU usage
- Per interface network usage
- Ping heartbeats for each machine
Zenoss Core supports CentOS/RHEL 5/6. Here we will show how to run Zenoss on a CentOS 6 minimal install. The Zenoss community team have done a great job with their installer script and packaging that make this process a breeze. First ensure that your hostname can be resolved on your network or is set to “localhost”. Next turn off your firewall by running:
iptables -F service iptables save
Once those two steps are complete its as easy as:
wget --no-check-certificate https://raw.github.com/zenoss/core-autodeploy/master/core-autodeploy-4.2.sh chmod +x core-autodeploy-4.2.sh ./core-autodeploy-4.2.sh
Once the install is complete we are ready to install and configure our bare metal machines for SNMP.
Install and configure snmpd
In order to install snmpd on CentOS/RHEL we just need to run the following on each machine to be monitored:
yum install net-snmp net-snmp-utils
After install is completed we need to open up our snmpd server to allow external requests to access its data. The config file for snmpd lives at /etc/snmp/snmpd.conf. We will edit that file as follows for each host to be monitored:
rocommunity public rwcommunity private syslocation MyDataCenterRack1 syscontact My Name <email@example.com>
- rocommunity is an arbitrary name given to group of users/monitors that will only need to read data from snmpd. Keep this setting the same across all hosts in the same private cloud.
- rwcommunity acts as a sort of security mechanism for changing data through snmpd. Only users with the rwcommunity will be able to change the configuration of a host through SNMP client calls. Keep this setting the same across all hosts in the same private cloud.
- syslocation defines an arbitrary name for where the particular server resides physically
- syscontact defines the name and email of the person who should be contacted in case of an issue with the machine
After configuration is complete lets ensure that the snmpd server is running and that its starts as part of the boot process:
service snmpd start chkconfig snmpd on
When you first try to access your Zenoss server in a browser at http://<server-ip>:8080, you should be presented with a wizard that takes you through:
- Changing the admin password
- Setting up your first user
- Discovering new devices
When you have finished steps 1 and 2 the third step will look like:
In this part of the wizard you’ll have to enter the IPs ranges or individual IPs of your Eucalyptus components that we have configured snmpd on in the above steps. Once you have entered the proper IPs and continued the process the wizard will discover your devices and put them in a special area for newly discovered devices. You can access them by clicking the Infrastructure button on the top nav bar then clicking “Discovered” on the left.
What we want to do now is setup a special type of container for our machines that defines more closely what we want to monitor on our machines.
To do this we will setup a new Device Class that is a sub class of Server/Linux called “Eucalyptus”:
- Click the arrow next to Server on the left side of the Infrastructure tab. Then click the Linux device type.
- Now click the plus sign at the bottom of that pane.
- Add a new device type called “Eucalyptus”
In order to get our Eucalyptus machines to be monitored fully we will need to add more modeler plugins to that device type:
- Click on the newly created “Eucalyptus” device type.
- Click the details arrow at the top right of that pane
- Click on “Modeler Plugins”
- Add all of the modeler plugins from the left side of the screen to the right side of the screen
- Click save
Next we need to tell Zenoss which processes we’d like to monitor on this device type:
- On the second tiered nav bar from the top, click Processes
- Add the following process types (httpd, postgres, eucalyptus-cloud, tgtd, kvm) using the following procedure:
- Click the plus sign at the bottom left, then Add Process
- Name the process
Now that we have our generic monitoring (interfaces, disk, memory, cpu) and our process specific monitoring complete we can move the devices we previously discovered into the “Eucalyptus” device type:
- Click on the “Discovered” device type in the left pane of the “Infrastructure” tab
- Drag all of the Eucalyptus components from the right pane to the left pane under the device type “Eucalyptus”
- Now highlight all the devices in the right pane
- Click the gear wheel in the bottom left of the interface and click Model device. This step ensures your devices are being monitored as Eucalyptus devices.
After an hour or so of monitoring your cloud you should be able to see useful data about its current state. Moving forward you can tune and enhance your cloud as is required by your workload without guessing. You now have real data to back up your optimizations: