Amazon's firewall setting are by default very restrictive. You need to set up a "security group" that will allow all your machines to communicate with each other. The "kluster" tools have a script that will do this for you. Type kl-create-sg mycluster
. (Note: here, "mycluster" is an arbitrary name that you can change to whatever you want.) You should see output like the following:
kl-create-sg mycluster /root/kluster-code2/bin/kl-create-sg: Creating security group mycluster GROUP sg-50de323b mycluster Security group for cluster 'mycluster' /root/kluster-code2/bin/kl-create-sg: Adding permissions for security group mycluster GROUP mycluster PERMISSION mycluster ALLOWS tcp 0 65535 FROM USER NAME mycluster ingress GROUP mycluster PERMISSION mycluster ALLOWS udp 0 65535 FROM USER NAME mycluster ingress GROUP mycluster PERMISSION mycluster ALLOWS icmp -1 -1 FROM USER NAME mycluster ingress GROUP mycluster PERMISSION mycluster ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 ingress
/root/kluster-code2/bin/kl-create-sg: Done adding permissions.
Look at the script bin/kl-create-sg
and see what it is doing. It just has some very simple calls to commands such as ec2-create-group and ec2-authorize-group. If at some point you want to delete this security group, you can do so with
ec2-delete-group mycluster
Next: Creating a key-pair
Up: Kluster Wiki
Wiki: CreatingKeyPair
Wiki: GettingYourKeys
Wiki: Home
"Amazon's firewall setting are by default very restrictive", but in a good way. This security group is a tad too promiscuous...
This error message is shown when using -> kl-create-sg mycluster
Client.InvalidParameterValue: Invalid security group description. Valid descriptions are strings less than 256 characters from the following set: a-zA-Z0-9. _-:/()#,@[]+=&;{}!$* (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 53793411-6c62-416a-9a24-8838e6abd89e)
but this works
ec2-create-group mycluster -d "Security group for cluster my cluster"
Last edit: Joseph D 2015-06-04
That is very odd, as "mycluster" is from within that set. Add -w to
the head of the bash script so you can see exactly what ec2-* commands
it is invoking.
Incidentally, the "kluster" repository and wiki may possibly be in a
slightly inconsistent state right now, as we have been making some
changes to make it work on Debian 7 (and for other reasons) and may
not have been keeping the wiki fully up to date.
Carl, I'm hoping that by the end of the workshop we can distil
everything down to a "clean" recipe so that someone could in principle
take the "kluster" repo and the instructions and replicate what we
did. Obviously this will be a nonzero amount of work, but it seems
like the right thing to do.
Dan
On Thu, Jun 4, 2015 at 3:42 PM, Joseph D josephd112@users.sf.net wrote: