Home modified by David Tulga

David Tulga — Tue, 11 Mar 2014 17:57:50 -0000
--- v1
+++ v2
@@ -1,8 +1,95 @@
-Welcome to your wiki!

-This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].
+RainforestCluster 0.9 Beta (2014-3-10)
+========
+by David Tulga
+Originally created for the Wall Lab at Harvard CBMI

-The wiki uses [Markdown](/p/rainforestcluster/wiki/markdown_syntax/) syntax.
+See [INSTALL] for install instructions
+See [AMI_INSTALL] for AMI creation and customization instructions
+See [LICENSE] for license text (Open Source MIT-style license)
+
+
+Description
+--------
+
+This is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility, minimizing costs and maximizing workflow speed. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run Cosmos, GenomeKey, or other computational pipelines generically. It is also able to optimize for the use of Spot Instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises.  It also provides pre-installed features such as GlusterFS distributed filesystems, RAID0 /scratch, passwordless ssh, and automatic cluster connection, for ease of use and maximum processing speed for the computational tools. 
+
+
+Usage
+--------
+
+rainforestcluster <global_command> <command_args>
+rfcluster <global_command> <command_args>
+
+rainforestcluster <cluster_command> <cluster_name> <command_args>
+rfcluster <cluster_command> <cluster_name> <command_args>
+
+
+Global Commands
+--------
+(All commands can be as command or --command)
+
+**help** : Print general help
+
+**version** : Print Current Version
+
+**list listclusters** : Lists all the clusters manageable by rainforestcluster
+    -l --long : Lists extra information about each cluster, such as state, size, etc.
+
+== Cluster Commands ==
+(All commands must be preceded with the cluster name, and can also be as command or --command)
+
+**start** <cluster_parameters> : start a cluster
+    --cluster -c --template : Specify a cluster template other than the default
+    --loadbalance <min> <max>
+    --loadbalance <max> (where min = config default or 0 if not set)
+    --loadbalance (will use the config defaults) : Will start a loadbalancer loop to change the cluster dynamically based on load, the min and max parameters are the number of nodes (0 means only the master is active)
+
+**status** : get cluster status
+
+**sshmaster** <ssh options> : ssh to the master instance for the cluster (ALSO: sshsuper)
+
+**sshto** <machine> (<ssh_options>) : ssh to a given machine name in the cluster (ALSO: ssh)
+
+**put** <local file> (<optional: remote location>) : scp a local file to the cluster, defaults to ... (master instance only)
+
+**get** <remote file> (<optional: local location>) : scp a remote file to the local computer, defaults to the current directory (master instance only)
+
+**addnodes** <number of nodes> : Add the specified number of nodes to the cluster
+
+**removenodes** <number of nodes> (OR: <list of node names separated by a space>): Remove either the specified number of nodes (chosen by least active first), or the nodes given by name (same as terminate --machine <...>)
+
+**loadbalance** <min> <max> : **loadbalance** <max> : **loadbalance** : Will start a loadbalancer loop to change the cluster size as needed due to job load
+
+**terminate** : fully terminate a cluster, removing additional security groups, spot requests, instances, etc. (will not delete any ebs volumes (except instance boot volumes) or S3 data.)
+    --nodes_only : Only terminate all nodes
+    --node --machine -m <machine_name(s)> : or : <machine name(s)> : Only terminate the given machine(s)
+
+Simultaneous/thread safety: Help and version at any time, sshmaster, put, and get are safe to run simulataneously once a cluster is fully active,
+start and terminate can only run by themselves with no other commands at the same time, and add/remove/loadbalance allow other commands, but only one in this group at a time.
+
+
+Examples
+--------
+rainforestcluster help
+rfcluster help
+
+rfcluster start test
+rfcluster status test
+rfcluster sshmaster test
+rfcluster sshmaster test -YC -L 8080:localhost:8080
+rfcluster put test /home/user/file1.txt /home/ubuntu/
+rfcluster put test file2.txt                                  (Defaults to placing it in either the custom user or ubuntu's home directory)
+rfcluster get test /gluster/gv0/file3.txt /home/user/Downloads/
+rfcluster get test /home/ubuntu/file4.txt               (Defaults to placing it the current directory)
+rfcluster get test "~/file4.txt"                            (Remote ~ must be in quotes)
+rfcluster addnodes test 2
+rfcluster removenodes test 2
+rfcluster removenodes test node005 node006
+rfcluster loadbalance test
+rfcluster loadbalance test 0 10
+rfcluster terminate test
+

 [[members limit=20]]
 [[download_button]]
Home modified by David Tulga

David Tulga — Sun, 09 Mar 2014 19:57:52 -0000
Welcome to your wiki!
This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].
The wiki uses Markdown syntax.
Project Members:

David Tulga (admin)
Recent changes to Home

Home modified by David Tulga

Home modified by David Tulga

Project Members:
