Creating a VM to mimic EC2

I spend a lot of time working with Amazon’s EC2 (Elastic Compute Cloud) virtual servers. Configuring a new server takes up a surprisingly large amount of time when done manually, and so I use Puppet to automate the process. This works pretty well, but after creating lots of new configuration code I always think it’s worth testing the complete installation on a local Virtual Machine. I’ve built a VM which looks quite a lot like an empty Ubuntu 12.04 cloud image, but have always worried that the CD installation might have a few key differences. Here’s a way to work round some of these.

EC2 instances use a different set of sources for APT, so the first step is to sync these up. Wipe the current /etc/apt/sources.list, and replace with an example from the cloud image you’re trying to emulate. I used the following lines:

deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise main
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise main
deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates main
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates main

deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise universe
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise universe
deb http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates universe
deb-src http://eu-west-1.ec2.archive.ubuntu.com/ubuntu/ precise-updates universe

deb http://security.ubuntu.com/ubuntu precise-security main
deb-src http://security.ubuntu.com/ubuntu precise-security main
deb http://security.ubuntu.com/ubuntu precise-security universe
deb-src http://security.ubuntu.com/ubuntu precise-security universe

Now a quick apt-get update will get you installing the same packages as EC2. The next logical step is to reinstall all the current packages to match. I thought this would be pretty easy, and went for the following invocation:

dpkg --list |
grep ^ii |
awk '{ print($2) }' |
xargs apt-get install --reinstall

This works as follows:

  1. Get the current list of packages
  2. Return only those which are actually installed
  3. Drop everything in the output but the second column (the column containing the package name)
  4. Pipe into xargs which in turn calls APT.

This got things moving and I’d soon downloaded 397 fresh packages. Unfortunately things then stopped with the (un)helpful message xargs: apt-get: terminated by signal 11. I tried calling APT separately, and got slightly further: Segmentation fault (core dumped). Did I say this got me further?

Reinstalling packages individually seemed to work, so I developed the theory that APT chokes when fed too many packages in one go. This bug seemed to be related. I worked around it with a simple modification to the xargs command to take ten packages at a time:

dpkg --list |
grep ^ii |
awk '{ print($2) }' |
xargs -L 10 apt-get install --reinstall

I decided to take another look at the Launchpad bug report linked earlier. This states:

When trying to –reinstall multiple packages with apt-get, it crashes with a segmentation fault. The command line: apt-get –reinstall install <2-or-more-packages...>

Having used Ubuntu for free for many years I thought it might be about time I gave something back to the community, so I thought I’d take another look. I’ve posted a comment with some more information after getting rather more involved with GDB and apport then I would have liked!

I genuinely don’t know if there is any difference between the packages for EC2 and those in the standard repositories (I rather suspect there isn’t), but if there is then I definitely don’t want to find out about it when trying to fix an urgent problem.

This entry was posted in Computing. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>