Rescuing Amazon EC2

I’ve recently been trying to install OpenVZ on an Amazon EC2 instance. It’s a process fraught with peril and I’ve locked myself out of the server more than once. Here are the steps to recover control:

  • Find the instance X1 that is not responding, make sure it is stopped. Note which volume is attached to the instance (look in the lower section of the page, click on the ‘Root Vol’ device and note the number).
  • Go to the Volumes section, find the volume V1 that is attached to that instance and detach it from the instance.
  • If you don’t have a spare instance to spin up, then go to the AMI section and spin up a small instance of whatever-you-fancy. It’s only temporary for fixing the disk.
  • Stop this instance, go to the Volumes section, select the volume to fix V1 and attach it to the spare machine, usually on /dev/sdf but this may vary.
  • Now back to the Instances section, spin up our spare temporary instance and, when running, login through SSH
  • Now we have the troublesome volume, V1, attached to /dev/sdf (or /dev/xsdf more likely) and we can mount that on /mnt/ (sudo mount /dev/xsdf /mnt) and make whatever changes we need to make.
  • Now we unmount it (sudo umount /mnt) and back in the Amazon control panel, go to the Volumes section and detach the volume from the instance.
  • Next reattach it to the original instance X1 and spin that up.
  • Pop the champers!
    You can of course avoid all this if you take a snapshot before making potentially destructive changes to the system!