Virtualised Development Environments on a Mac

Published 22:23 on 06 September, 2009

Recently, I bought myself a brand new MacBook Pro, which gave me the perfect opportunity to clean up my development environment. Since I’ve started doing all of my development on virtual machines, I began thinking about my development workflow: In theory, I should be able to model the perfect server environment virtually.

I wasn’t sure what environment I wanted to end up with, but I had a good idea what basic virtual machines I wanted as a starting layout.

The Proposed Environment

A basic, well structured development environment should include:

Development Server -- Where all development takes place. This will probably have more detailed logs, development dependencies, and a high likelihood of being broken at any given moment in time. Bugs that occur here are likely to be frequent.
Staging Server -- A duplicate of the production server that allows testing of build scripts, packages etc. before they are pushed to production. Bugs that occur here are likely to be less frequent and flagged by specific testing.
Production Server -- The live environment. By the time the code reaches this server, it should have gone through development testing and staging testing. This means bugs should be at a bare minimum.

If bugs arise on either the staging server or the production server, the fixes should be made on the development server and pushed to staging and then production. No code changes should happen on staging or production.

We could, of course, add more stages between development and live (a quality assurance (QA) server, for instance), and we could even branch the environment to allow for a continuous integration (CI) server (the product of an automated build, and including automated testing; see Wikipedia’s Continuous Integration page for more information).

In my specific case, the environment should be structured thus:

Development Server -- The development virtual machine I've been using for the past few months. This VM shares key folders with Mac OS X so that I get the pleasure of using Mac based tools for development (see: TextMate).
Staging Server -- A duplicate of the production server (or as close to as possible) in VM format. This should be the target for my build scripts, and where I run integration tests.
Production Server -- My live web server. Only functioning, tested code should ever be pushed here.

So, rather than developing with just the one virtual machine, it’s clear I should be moving forward with two virtual machines (at least) – one as the development server, and one as the staging server.

Linked Clones

Once I’ve built a good master virtual machine as a source – and set it all up as a basic server, including all the dependencies and packages I want – I could create duplicates for each server. The drawback to that method is the considerable amount of disk space the independent VMs would take up.

The second option would be to branch snapshots on the master, setting up one branch as the development server, and another as the staging server. However, this would mean that I would have no way of running the two servers simultaneously, which would be a terrible trudge when attempting to test a build.

The best solution would be to build linked clones. A linked clone is a copy of a virtual machine that shares virtual disks with the parent virtual machine in an ongoing manner. This conserves disk space, and allows multiple virtual machines to use the same software installation. It also means we’ll be able to run the two VMs simultaneously. However, VMWare Fusion (the Mac version of VMWare) doesn’t actually allow this functionality through the interface; so it has to be done manually.

When you take a snapshot of a VM, the original virtual disk becomes read-only and a new Copy-On-Write (COW) disk is created. To create a linked clone manually, we’re going to set up a virtual machine, then create new ones using that as the base disk. We then snapshot the new VMs so they never try to modify the base disk.

Note: I pretty much followed the “HOWTO: Manual Linked Cloning in VMWare Fusion” article on the VMWare Communites site. I’ve recreated the steps below to allow illustration of my particular implementation, and to hopefully save you flicking backwards and forwards between articles.

Building the environment

The first thing we need to do is create a master VM. This will be our base disk, and the root of all evil… Uh… I mean… The root of all awesome.

Create the master VM

In my particular instance I’m creating web servers, so I’m going to build an Ubuntu VM and base the clones on that. To do this, I’d recommend following Brad Wright’s excellent Ubuntu VM build tutorial, right up to the point where you’ve finished installing VMWare Tools – we don’t want to start sharing folders on our base build because we’ll do that on the development server VM only.

The next step is to set up all the common packages, dependencies, configurations, and applications on your base VM so that you have the minimum to set up on your clones (and so they’re as close to one another as possible at the outset; after all, that’s part of the reason we’re doing this).

In my case, I set up the following:

Apache
PHP
MySQL
Python
Django
CouchDB
Twisted

I also made sure I’d configured them all as I wanted them – including default site layout for Apache – and included interdependencies like MySQL-Python.

Once this is complete, we’ve got a suitable master VM for our linked clones.

Finally, make sure you back up the .vmwarevm package for your master VM. We’re going to be editing the package contents, and deleting a bunch of stuff; plus if anything gets hosed in the future, you’ll be able to restore the master and start over. I’d recommend creating an archive of the .vmware package and storing it somewhere safe.

Prepare the master VM for cloning

The only thing we want to keep from the master is the virtual disk, since the rest of the VM is inconsequential to our linked clones. So, once you’ve definitely made a back-up of the full VM, let’s change directory to where our VMs are stored, copy out the .vmdk files (virtual machine disk), and delete the master bundle:

$ cd /Users/timbo/vms
$ ls
master.vmwarevm
$ mkdir master
$ ls
master  master.vmwarevm
$ mv master.vmwarevm/*.vmdk master/
$ rm -f master.vmwarevm
$ ls master
master-s001.vmdk  master-s002.vmdk  master-s003.vmdk  master-s004.vmdk  master-s005.vmdk  master-s006.vmdk  master.vmdk

Next we need to update the the virtual disk to reflect our changed location and to remove it’s UUID (Universally Unique Identifier). To do that, we need to edit the root .vmdk file:

$ cd master
$ mate master.vmdk

I’m using TextMate as my editor of choice, but you could easily use vi, vim, nano, BBEdit etc.

You should see a file that looks something like this:

# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=96100eeb
parentCID=ffffffff
createType="twoGbMaxExtentSparse"

# Extent description
RW 4192256 SPARSE "master-s001.vmdk"
RW 4192256 SPARSE "master-s002.vmdk"
RW 4192256 SPARSE "master-s003.vmdk"
RW 4192256 SPARSE "master-s004.vmdk"
RW 4192256 SPARSE "master-s005.vmdk"
RW 10240 SPARSE "master-s006.vmdk"

# The Disk Data Base 
#DDB

ddb.uuid = "60 00 C2 99 ab ce aa 92-d3 3e 55 a9 30 e7 c6 dd"
ddb.toolsVersion = "7462"
ddb.adapterType = "lsilogic"
ddb.geometry.sectors = "63"
ddb.geometry.heads = "255"
ddb.geometry.cylinders = "1305"
ddb.virtualHWVersion = "7"

We need to fix the paths of our .vmdk files so that they will work from other locations. Obviously, for the best results, we should probably use absolute paths so that they work from anywhere. I updated mine like this:

# Extent description
RW 4192256 SPARSE "/Users/timbo/vms/master/master-s001.vmdk"
RW 4192256 SPARSE "/Users/timbo/vms/master/master-s002.vmdk"
RW 4192256 SPARSE "/Users/timbo/vms/master/master-s003.vmdk"
RW 4192256 SPARSE "/Users/timbo/vms/master/master-s004.vmdk"
RW 4192256 SPARSE "/Users/timbo/vms/master/master-s005.vmdk"
RW 10240 SPARSE "/Users/timbo/vms/master/master-s006.vmdk"

The next thing you should do is remove the UUID line, either by commenting it with a #, or by deleting it entirely:

ddb.uuid = "60 00 C2 99 ab ce aa 92-d3 3e 55 a9 30 e7 c6 dd"

Once these changes have been made, save the file and exit back to the command line.

The final thing we should do is write protect these files – make sure you update your path appropriately:

chmod a-w /Users/timbo/vms/master/*.vmdk

Our master is now totally prepared for cloning, and can be used repeatedly. In my instance, this means I can now clone it twice; once for the development VM, and once for the staging VM (since it’s no longer a full VM by itself).

Create a clone

To create a clone, we need to create the VM in VMWare Fusion. Once you’re back in the VMWare GUI, do the following:

Push cmd + N, or select File > New from the menu, to create a new virtual machine.
Click "Continue without disk".
Select "Create a custom virtual machine" and click "Continue".
Select the appropriate guest OS type (in my case, Linux/Ubuntu) and click "Continue".
Click "Customize Settings".
Choose a name and location for your clone VM. I chose to call mine "child" and saved it in my ~/vms directory.
You should now be looking at the settings of your new VM. Set it up with the same basic settings as you did for the master (256MB RAM, printers disabled), but leave the HDD alone because…
Edit the HDD settings and delete the disk device by clicking the "-" when the device is selected in the list.
Go back to your terminal and delete the .vmdk files in the new VM package:
```
$ cd /Users/timbo/vms/child.vmwarevm
$ rm -f *.vmdk
```
Now copy the root .vmdk file from your master (the one we edited the paths in earlier) to your clone package:
```
$ cd /Users/timbo/vms
$ cp master/master.vmdk child.vmwarevm/
```
We need a copy because VMWare will create a .lck file at the same level as this file. With no copy, all the clones would be attempting to use the same .lck file simultaneously, which would prevent them running in parallel.</li>
Back in the Hard Disk settings pane, press the + button to add a new HDD device.
Select "Choose existing disk..." in the "File name" drop-down, and choose the copied metadata file (the one in child.vmwarevm, not the one in the master folder) and uncheck the checkbox -- we're happy with the file where it is, we don't want to copy or move it since we've already copied it manually (VMWare copies all the .vmdk files over, and that wouldn't save us any disk space at all).
Finally, create a snapshot of your new VM. This tells Fusion not to try to write to the original disk. I named mine "Base" with the comment "Don't delete!". Even if you run without the snapshot, because of the read-only permissions on the .vmdk files you shouldn't be able to change the original virtual disk, but the clone virtual machine (i.e. not the master) will not be happy.
Run the clone.
Since this is a clone of your master, log in using the same user(s).
Run the interface configurator:
```
$ ifconfig
```
If the eth0 interface is showing, skip ahead to step 19, otherwise…
To fix this problem, we just need to truncate the udev persistent network rules file:
```
$ sudo cp /dev/null /etc/udev/rules.d/70-persistent-net.rules
```
Reboot the VM to regenerate the rules:
```
$ sudo reboot
```
Now wait for your VM to reboot and log back in again.
Run the interface configurator:
```
$ ifconfig
```
Make a note of the IP address of the eth0 interface, alongside inet addr: since this is the new address for your clone.
Finally, update the hostname of your clone VM by editing and saving the /etc/hostname file:
```
$ sudo vim /etc/hostname
```
Restart the hostname service:
```
$ sudo /etc/init.d/hostname.sh start
```
Use your clone VM as normal.

Playing nice

A Power User's Guide to VMWare Fusion

$ defaults write com.vmware.fusion fluxCapacitor -bool YES

$ /Library/Application\ Support/VMware\ Fusion/vmrun start /Users/timbo/vms/child.vmwarevm/child.vmx nogui

$PATH

.bash_profile

export PATH="$PATH:/Library/Application Support/VMware Fusion"

vmrun

$ vmrun start /Users/timbo/vms/child.vmwarevm/child.vmx nogui

Shared folders

Brad Wright's excellent Ubuntu VM build tutorial

vmrun

$ vmrun addSharedFolder /Users/timbo/vms/child.vmwarevm/child.vmx myfolder /Users/timbo/projects/myfolder

vmrun

The Proposed Environment

Linked Clones

Building the environment

Create the master VM

Prepare the master VM for cloning

Create a clone

Playing nice

Shared folders

Summary