Clusters of Development Virtual Machines

Since I'm working with clusters of virtual machines hosting my development environment on my local workstation, I've come up with method to simplify login, the remote-execution of commands and the exchange of data between the host and the virtual machine instances. This post will show you my way of keeping structure in all the virtual machine configurations, and a couple of shell-functions helping me using virtual machine instances efficiently.

The basic idea is to maintain a dedicated directory for each virtual machine and its related file, like the login key or the metadata for the configuration management. Following you can see a listing of one of the virtual machine directories:

$ ls /srv/vms/lxdev01.devops.org
chef_attributes.json  
chef_config.rb  
disk.img  
keys/  
libvirt_instance.xml  
provision.sh*  
ssh_config

As I have written before in my post "Bridging a Host-Internal Network for Virtual Machines", I like to use meaningful hostnames. To avoid confusion each of my development virtual machines is stored in an directory called like its FQDN. When I start to setup a new test virtual machine, I begin with copying one of my golden images created before (which was explained in my blog post "Installing KVM Virtual Machines"). Except of the disk image and the libvirt configuration, you can see a couple of other helper files for a virtual machine in the listing above. We will cover ssh_config and the keys/ directory in this post, and the Chef configuration management related files in future.

The following functions I'm keeping in an file called helpers.sh, which I source into my shell environment when needed, enables remote interaction with the virtual machine store in the current working directory:

# Login to the virtual machine, or execute a command, e.g.:
#
#    vmssh 'ls /tmp'
#
function vmssh() { ssh -qt -F $PWD/ssh_config instance $@; }

# Upload a file into the virtual machine, e.g.:
#
#    vmput /path/to/local/file /tmp
#
function vmput() { scp -F $PWD/ssh_config $1 instance:$2; }

# Download a file from the virtual machine, e.g.:
#
#    vmget /path/in/vm /tmp
#
function vmget() { scp -F $PWD/ssh_config instance:$1 $2; }

# Sync a locael directory to the virtual machine, e.g:
#
#   vmsync /local/path /tmp
#
function vmsync() { 
  rsync -va -L --exclude '.git' --exclude '.gitignore' \
        -e "ssh -F $PWD/ssh_config" $1 instance:$2;  
}

All the functions us the SSH configuration file ssh_config stored in the virtual machines directory, defining the login account name and the virtual machine IP-address:

Host instance
 User devops
 HostName 10.1.1.11
 IdentityFile /srv/vms/lxdev01.devops.gsi.de/keys/id_rsa
 UserKnownHostsFile /dev/null
 StrictHostKeyChecking no

Another ingredient in this file is the 'IdentityFile', the public part of an SSH-key-pair stored in the sub-directory keys/. You can create a new password-less SSH key and uploaded it into the virtual machine using the following commands:

$ mkdir keys; ssh-keygen -q -t rsa -b 2048 -N '' -f keys/id_rsa
$ vmssh 'mkdir -p -m 0700 $HOME/.ssh'
$ vmput keys/id_rsa.pub .ssh/authorized_keys

I think the vmssh, vmput and vmget functions are obvious. If the Rsync utility is installed in the virtual machine and on your host, you can sync entire directory structures between the host and the instance using vmsync.