CreatingInstance

Creating a Debian Linux Instance

We have chosen to base these instructions on Debian Linux. Running on a Red Hat distribution or Amazon's equivalents is probably easy too, but our scripts would not work as written.

The first step is to choose a suitable "image" to customize. At this point you probably should also choose an "availability zone" (read: data center) to run in, since the images are specific to different availability zones. I chose us-east-1c. I chose a Debian image from this list of Debian images; I chose the image ami-e0efab88. There was nothing very special about this image, but it had a couple of features:

• It is a 64 bit, not 32 bit, image.
• It is "ebs-backed", not "instance-store backed".
• At the time of writing (Jun 4, 2015) it was the latest Debian version providing gridengine

EBS-backed means that you can stop and start the image and it won't lose the data; this is generally more convenient. Now, if you want to follow these instructions exactly you should choose exactly this image, because this recipe might not work for other images.

We will be customizing this image to create the image for our cluster. It would have been quite possible for us to release a customized image directly to save you some work, but we wanted to

• document the process of creating this image
• give you some experience with EC2 tools and Debian administration

To create your instance, do as follows:

ec2-run-instances ami-e0efab88 -g mycluster -k mycluster -z us-east-1c -t c3.large

The option -g mycluster puts the new instance in the security group "mycluster". The option "-k mycluster" means that it will use the keypair called "mycluster" (this is a separate type of object to which we assigned the same name). The -z option specifies the "availability zone" (presumably, data-center), and -t c3.large specifies the type of machine ("instance type"). The output should look like this:

# ec2-run-instances ami-e0efab88 -g mycluster -k mycluster -z us-east-1c -t c3.large 
RESERVATION r-9aa24ae7  167806836050    mycluster
INSTANCE    i-7877a019  ami-e0efab88            pending mycluster   0       c3.large    2015-06-04T20:07:58+0000    us-east-1c  aki-4e7d9527          monitoring-disabled                 ebs                 hvm xen     sg-3decf455 default false

The output tells you various things about the instance you just launched. The format is easily parse-able by a script. The fields can best be extracted by e.g. cut -f 2 because they are tab-separated. Copy and paste the instance-id; in this case, it is i-7877a019. Then use the command ec2din i-7877a019 to get the information again:

# ec2din i-7877a019
RESERVATION r-9aa24ae7  167806836050    mycluster
INSTANCE    i-7877a019  ami-e0efab88    ec2-54-235-5-54.compute-1.amazonaws.com domU-12-31-39-04-30-B3.compute-1.internal   running mycluster   0       c3.large    2015-06-04T20:07:58+0000    us-east-1c  aki-4e7d9527            monitoring-disabled 54.235.5.54 10.240.55.65          ebs                 hvm xen     sg-3decf455 default false
BLOCKDEVICE /dev/sda    vol-51ca3b22    2015-06-04T20:08:03.000Z    true

Assuming you waited a few seconds for the instance to be assigned an address, the output should look like the above. Copy the address (ec2-54-235-5-54.compute-1.amazonaws.com in this case) and ssh to it as follows (you may have to wait a minute before it is ready):

# ssh -i ~/.ssh/mycluster.pem admin@ec2-54-235-5-54.compute-1.amazonaws.com
The authenticity of host 'ec2-54-235-5-54.compute-1.amazonaws.com (54.235.5.54)' can't be established.
RSA key fingerprint is d5:50:bd:8e:23:eb:14:3f:cf:15:87:42:0b:bf:e2:60.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-54-235-5-54.compute-1.amazonaws.com,54.235.5.54' (RSA) to the list of known hosts.
<snip>
admin@domU-12-31-39-04-30-B3:~#

Note that you have to ssh into the machine as admin at first, as root access through ssh is initially disabled. In order to get a prompt as root you can now do:

# sudo bash

If you don't like having to type "yes" at this point, you can set the following at the top of your file ~/.ssh/config on your local machine (create the file if it doesn't exist):

StrictHostKeyChecking no

The longer you leave this instance running the more money Amazon will charge you. You can stop this instance with the command ec2stop i-7877a019 and start it again with ec2start i-7877a019. When you want to kill it permanently (and this will lose anything you have written to its disk) you can kill it with

ec2kill i-7877a019

(but don't do this now!) It is useful to give your instances names so you don't forget what they are. From your local machine, type the following (using your own instance id)

ec2tag i-7877a019 --tag Name=customize_phase_1

Previous: Creating a key-pair
Next: Customizing your Image (Phase 1)
Up: Kluster Wiki

[Tal Weiss]

"ami-9e95e8f7" doesn't exist. In parts of this page you used "ami-e00df089" which works.

[Tal Weiss]

"m1.large" is a bad choice. It is old, which means less powerful and more expensive than newer types!

[Tal Weiss]

Log in as "admin@" was denied. Log in as "root@" was successful. Looking at "/etc/passwd" shows there is no admin on this instance.