AMI_INSTALL modified by David Tulga

David Tulga — Tue, 11 Mar 2014 18:12:59 -0000

--- v1
+++ v2
@@ -1,4 +1,3 @@
-
 AMI Installation/Creation/Customization/Upgrade Instructions
 ========

@@ -20,7 +19,7 @@
     rflib.py , \_\_init\_\_.py , and any remote plugins you are using to /etc/rfcluster_remote_plugins/
 Simply follow the customization instructions above to save a new AMI with these changes.

-If ThunderstormDistributor is upgraded, you will need to copy the new source code to ~/TSDistributor/
+If [ThunderstormDistributor](/p/thunderstormdistributor/) is upgraded, you will need to copy the new source code to ~/TSDistributor/
 Then, run the compile_install.bash script under ~/TSDistributor-Release/
 And copy all files from ~/TSDistributor/tsstatswww/ to /tsstatswww/ to update the stats viewer

@@ -43,7 +42,7 @@
 scp -i ~/.ssh/your_amazon_ec2_key.pem __init__.py ubuntu@ec2-XX-XX-XX-XX.compute-1.amazonaws.com:~/
 ~~~~~~~~

-To install ThunderstormDistributor, you will need to upload the source code to the instance: (also includes all stats viewer files)
+To install [ThunderstormDistributor](/p/thunderstormdistributor/), you will need to upload the source code to the instance: (also includes all stats viewer files)

 ~~~~~~~~
 :::bash
@@ -112,5 +111,4 @@
 ~~~~~~~~

-Optionally instead of the default install, you can also look at the ThunderstormDistributor install instructions for a custom install and/or configuration.
-
+Optionally instead of the default install, you can also look at the [ThunderstormDistributor install instructions](/p/thunderstormdistributor/wiki/INSTALL/) for a custom install and/or configuration.

AMI_INSTALL modified by David Tulga

David Tulga — Tue, 11 Mar 2014 18:08:51 -0000

AMI Installation/Creation/Customization/Upgrade Instructions

To customize or upgrade an already existing RainforestCluster AMI:

Start an instance with Amazon's AWS Management console or command line tools using the AMI you want to customize (Paravirtual/HVM). Any instance type will work, even a t1.micro.
Then, install or customize any programs you want on this machine, and save this machine as a new AMI.
Finally, set this AMI ID in your config.ini file and you're done!

To upgrade a customized AMI to the latest version of RainforestCluster:

If RainforestCluster is upgraded, you will need to copy the new version of:
rfcluster_remote.py to /etc/
rflib.py , __init__.py , and any remote plugins you are using to /etc/rfcluster_remote_plugins/
Simply follow the customization instructions above to save a new AMI with these changes.

If ThunderstormDistributor is upgraded, you will need to copy the new source code to ~/TSDistributor/
Then, run the compile_install.bash script under ~/TSDistributor-Release/
And copy all files from ~/TSDistributor/tsstatswww/ to /tsstatswww/ to update the stats viewer

To create a new RainforestCluster AMI from scratch:

If you want to create a new RainforestCluster AMI from scratch, for example to use a newer Linux version, etc. - then follow these instructions, starting from a template instance running on EC2:

Upload Necessary Files:

Remote scripts:

scp -i ~/.ssh/your_amazon_ec2_key.pem rfcluster_remote.py ubuntu@ec2-XX-XX-XX-XX.compute-1.amazonaws.com:~/
scp -i ~/.ssh/your_amazon_ec2_key.pem rflib.py ubuntu@ec2-XX-XX-XX-XX.compute-1.amazonaws.com:~/
scp -i ~/.ssh/your_amazon_ec2_key.pem __init__.py ubuntu@ec2-XX-XX-XX-XX.compute-1.amazonaws.com:~/

To install ThunderstormDistributor, you will need to upload the source code to the instance: (also includes all stats viewer files)

scp -i ~/.ssh/your_amazon_ec2_key.pem tsdistributor_....tar.gz ubuntu@ec2-XX-XX-XX-XX.compute-1.amazonaws.com:~/

Then run these commands, as root: (sudo su)

apt-get update -y
apt-get upgrade -y

Keep the same grub configuration if asked

apt-get install -y htop mdadm xfsprogs xfsdump build-essential sysbench iperf iotop

No configuration for postfix

add-apt-repository ppa:w-rouesnel/openssh-hpn -y
add-apt-repository ppa:semiosis/ubuntu-glusterfs-3.4 -y
apt-add-repository ppa:ubuntu-sdk-team/ppa -y

apt-get update -y
apt-get upgrade -y

apt-get install -y glusterfs-server glusterfs-client python-dev nginx qt5-default qtdeclarative5-dev

echo -e '\n# SSH HPN Settings\nHPNDisabled no\nTcpRcvBufPoll yes\nHPNBufferSize 8192\n' >> /etc/ssh/sshd_config

service ssh restart

echo -e '\n127.0.0.1 RainforestCluster-Beta\n\n# Entries for this and other machines in the cluster (Do not delete or modify these lines!)\n#RainforestCluster Start#\n#RainforestCluster End#\n' >> /etc/hosts

chmod 755 rfcluster_remote.py rflib.py __init__.py
chown root:root rfcluster_remote.py rflib.py __init__.py
mv rfcluster_remote.py /etc/
mkdir /etc/rfcluster_remote_plugins
mv rflib.py __init__.py /etc/rfcluster_remote_plugins/

tar -xzf tsdistributor_....tar.gz

TSDistributor-Release/compile_install.bash

cd TSDistributor/

cp global.conf /etc/tsdistributor/

cp nginx_default /etc/nginx/sites-available/default

mkdir /tsstatswww
cp tsstatswww/* /tsstatswww/

cp scripts/* /usr/bin/

Optionally instead of the default install, you can also look at the ThunderstormDistributor install instructions for a custom install and/or configuration.

Recent changes to AMI_INSTALL

AMI_INSTALL modified by David Tulga

AMI_INSTALL modified by David Tulga

AMI Installation/Creation/Customization/Upgrade Instructions

To customize or upgrade an already existing RainforestCluster AMI:

To upgrade a customized AMI to the latest version of RainforestCluster:

To create a new RainforestCluster AMI from scratch: