Bridging a Host-Internal Network for Virtual Machines

As system administrator I like to build infrastructure as homogeneous as possible. The company I'm working for uses KVM both for services virtualisation and in its private Cloud beneath OpenNebula. In either use-case libvirt is deployed as interface for KVM. To make it easy to migrate from the local development environment to the production installation, I prefer to use KVM and libvirt also on my workstation.

Meanwhile I'm feeling confident to say that with very little effort I'm able to recreate complex service setups involving clusters of virtual machines. Beginning with this post I will write about my experience of using libvirt/KVM, Chef and a couple of shell scripts to bootstrap minimal copies of the production systems for developing configuration management code.

All my development environments are enclosed into a host-internal network shared by all virtual machines, connected to the external world using a NAT. Fortunately libvirt makes it very easy setup such a NATted network bridge. Following you can see my configuration file (called libvirt_nat_bridge.xml in this example):

<network>
  <name>nat_bridge</name>
  <bridge name="nbr0" />
  <forward mode="nat"/>
  <domain name="devops.org"/>
  <ip address="10.1.1.1" netmask="255.255.255.0">
    <dhcp>
      <range start="10.1.1.20" end="10.1.1.254" />
      <host mac="02:FF:0A:0A:06:02" ip="10.1.1.2" name="lxdns01.devops.org"/>
      <host mac="02:FF:0A:0A:06:03" ip="10.1.1.3" name="lxcm01.devops.org"/>
      <host mac="02:FF:0A:0A:06:04" ip="10.1.1.4" name="lxrm01.devops.org"/>
      <host mac="02:FF:0A:0A:06:05" ip="10.1.1.5" name="lxb001.devops.org"/>
      <host mac="02:FF:0A:0A:06:06" ip="10.1.1.6" name="lxb002.devops.org"/>
      <host mac="02:FF:0A:0A:06:07" ip="10.1.1.7" name="lxb003.devops.org"/>
      <host mac="02:FF:0A:0A:06:08" ip="10.1.1.8" name="lxb004.devops.org"/>
      <host mac="02:FF:0A:0A:06:09" ip="10.1.1.9" name="lxmon01.devops.org"/>
      <host mac="02:FF:0A:0A:06:0A" ip="10.1.1.10" name="lxcfs01.devops.org"/>
      <host mac="02:FF:0A:0A:06:0B" ip="10.1.1.11" name="lxdev01.devops.org"/>
      <host mac="02:FF:0A:0A:06:0C" ip="10.1.1.12" name="lxdev02.devops.org"/>
      <host mac="02:FF:0A:0A:06:0D" ip="10.1.1.13" name="lxdev03.devops.org"/>
    </dhcp>
  </ip> 
</network>

The network-description above tells libvirt to create a network bridge called nbr0. (This involves the configuration of iptables to act as NAT and to routing IP-traffic. Furthermore it starts a dnsmasq process serving DHCP and DNS resolution for the virtual machines.)

Setup such a network-configuration using virsh, the libvirt command-line-interface:

$ virsh net-create libvirt_nat_bridge.xml

I like to have a predefined set of host names for my test systems along with associated IPs, in order to always find the configuration management system (e.g. Chef) named "lxcm01" and a resource management system (like GridEngine or Condor) as "lxrm01".

When writing the libvirt configuration file for the virtual machine you can assign a specific IP pair using the associated MAC-address, like:

<devices>
...
  <interface type="bridge">
    <source bridge="nbr0"/>
    <mac address="02:FF:0A:0A:06:0B"/>
  </interface>
...
</devices>

Depending on the infrastructure you will be developing, it may be necessary to use a virtual machine to host a Bind instance to have all features of DNS. I will cover such a setup in anther post, as well as how to do port-forwarding to virtual machine instances.