Create an Amazon EC2 Machine Image from Scratch (from a loopback file ie.)
A brief summary of our objective and method
Amazon’s AWS (an IaaS - Infrastructure as a service) platform is built on Xen virtualization. Our objective is to create an Amazon Machine Image (AMI) which we will use to launch “Instances” (virtual machines) in Amazon Web Services (AWS).
The method we will adopt is to prepare an AMI through a loopback file. We will first create a file in (any) CentOS machine with the dd command, format and mount the file, install CentOS 5.4 in the file (and other common tools/software/services). Then we will “bundle” and “upload” the image to Amazon S3 storage. Once the image is uploaded, we can register it as an AMI. We are now ready to deploy as many “instances” of the AMI as we please (or rather pay for!).
We need to install AMI Command Line tools from Amazon on the machine which we use to prepare the AMI. We also need to be prepared with an Account at Amazon with at least EC2 and S3. Amazon uses a simple form of symmetric key encryption so we need to generate and note our Private/Public keys, Acces/Secret keys. (ref: AWS User Guide, Amazon EC2 Concepts\EC2 Credential Concepts)
What this post does not cover
This post does not cover how to create a login at Amazon, generate, copy and secure the keys. It also does not cover how to download and install the AMI tools, provide the paths to the EC2 keys in a linux system. The method outlined below is not sufficient to make an image public, please refer Amazon EC2 User/Developer guides for more information.
Install CentOS in a Loopback File
Create a file of size between 1GB to 10GB (more than 10GB is not permitted by Amazon). Here, I am creating a file of size 8GB.
|# dd if=/dev/zero of=image.fs bs=1M count=8192|
Create an ext3 file system
|# mke2fs -F -j image.fs|
Create a mount point like /mnt/ec2-fs and mount the file through loopback
|# mount -o loop image.fs /mnt/ec2-fs|
Create a dev directory and populate it with minimal set of devices
|# mkdir /mnt/ec2-fs/dev |
# /sbin/MAKEDEV -d /mnt/ec2-fs/dev -x console
# /sbin/MAKEDEV -d /mnt/ec2-fs/dev -x null
# /sbin/MAKEDEV -d /mnt/ec2-fs/dev -x zero
Create a few other folders
|# mkdir /mnt/ec2-fs/proc |
# mkdir /mnt/ec2-fs/etc
# mkdir /mnt/ec2-fs/var
# mkdir /mnt/ec2-fs/var/cache
# mkdir /mnt/ec2-fs/var/log
# mkdir /mnt/ec2-fs/var/lock
# mkdir /mnt/ec2-fs/var/lock/rpm
|/dev/sda1 / ext3 defaults 1 1 |
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
Mount the image’s proc device
|# mount –t proc none /mnt/ec2-fs/proc|
Create a yum configuration file: yum-ami.conf
#packages used/produced in the build but not released
#additional packages that may be useful
#additional packages that extend functionality of existing packages
#contrib - packages by Centos Users
Install the OS with all the common packages (routinely) needed on a Server, with the exclusion of specific servers like HTTPD, sendmail etc. (GNOME Desktop and X Window System may not needed in some environments, add more or remove some as per your requirement. You can use yum list as a reference.)
|yum -c yum-ami.conf --installroot=/mnt/ec2-fs groupinstall Base Editors "Legacy Software Development" "Legacy Software Support" Ruby "System Tools" "Text-based Internet" "Yum Utilities" "Development Libraries" "Development Tools" "GNOME Desktop Environment" "Graphical Internet" Java "X Window System"|
Note that we are doing a Base groupinstall, you may also choose to do a Core install with minimal set of components.
Install openssh (and any other/more sofware)
|yum –c yum-ami.conf --installroot=/mnt/ec2-fs/ install openssh*|
Configure the image
Disable services that you do not need (EC2 instances runs at level 4)
for i in cpuspeed pcscd messagebus restorecond mcstrans \
Start services that you need (EC2 runs at level 4)
for i in network rsyslog sshd; do
Create /mnt/ec2-fs/etc/sysconfig/network-scripts/ifcfg-eth0 and add
Set networking to start
Download, copy and probe all modules required by EC2 (which are not installed by yum. The modules may be downloaded from http://ec2-downloads.s3.amazonaws.com/ , or you may just boot up a public image and pick the modules from there!
copy to /mnt/ec2-fs/lib/modules
Probe the modules
/sbin/depmod –ae <module name>
Permit Root login without password (as per Amazon), or alternately if would like to simply permit root login
Set PermitRootLogin yes (if you want to)
or, PermitRootLogin without-password (as recommended by Amazon)
If you would like to permit root login use pwconv
|# chroot /mnt/ec2-fs /bin/sh |
# pwconv /etc/passwd
Create .bashrc file in /mnt/ec2-fs/root
# User specific aliases and functions
alias rm='rm -i'
# Source global definitions
Check if ~/.bash_profile is present, else create that as well. You may just copy from the base machine. Otherwise, you would stare into a prompt like “–bash-ver.num $” rather than the pleasant “user@hostname working directory #” prompt!
Set SELINUX to disabled
|vi /mnt/ec2-fs/etc/selinux/config |
Amazon uses Symmetric Key encryption, so the AMI instances should be able to download the public key from the metadata URL and add it to the ~/.ssh/authorized_keys file. Also, Amazon recommends to install AMI tools on the instances. Following should be added to the /mnt/ec2-fs/etc/rc.local file in the image.
# Update the Amazon EC2 AMI creation tools
# Fetch sshkey (public key)
Cleanup and unmount (cleanup can save a lot of space depending on what kind of install you did, if you installed a lot of stuff, then a clean up is so much recommended).
Unmount the loopback file after cleanup.
|# yum –c yum-ami.conf --installroot=/mnt/ec2-fs clean all |
# umount /mnt/ec2-fs/proc
# umount –d /mnt/ec2-fs
Bundle, Upload and Register the AMI
Bundle the image
|# ec2-bundle-image –i image.fs –k <path to private key> –c <path to certificate> –r x86_64 –u <AWS account number>|
Upload the image to your bucket in Amazon S3 storage
|The files bundled in earlier step would be found in /tmp |
# ec2-upload-bundle -b <you bucket name> -m image.fs.manifest.xml -a <your AWS access_key> -s <your AWS secret_key>
Register the image as an AMI
|ec2-register <your S3 bucket name>/image.fs.manifest.xml –n <image name of your choice>|
Note that we need not put any keys, or credentials in the image files, and we should not put such things in the image. If you wish to make your image public, then follow the procedures from EC2 user guide.
I use the Firefox plugin “Elasticfox” to manage EC2, you may also download and use it. Or you may administer from the AWS console available at aws.amazon.com , or you may also use the AMI Tools. If you are a developer, then there are the API tools and the Developer Guide.