RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. Originally it was developed as a different version for the Wall Lab at Harvard CBMI.
- Create Amazon EC2 custom clusters dynamically
- Add, remove, and loadbalance nodes
- Create large shared gluster volumes with spanning across multiple masters
- Distribute jobs efficiently using ThunderstormDistributor
- View machine, job, and cluster statistics on a web-based interface
Be the first to post a review of RainforestCluster!