RainforestCluster

RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. Originally it was developed as a different version for the Wall Lab at Harvard CBMI.

Features

Create Amazon EC2 custom clusters dynamically
Add, remove, and loadbalance nodes
Create large shared gluster volumes with spanning across multiple masters
Distribute jobs efficiently using ThunderstormDistributor
View machine, job, and cluster statistics on a web-based interface

Project Activity

See All Activity >

License

MIT License

Follow RainforestCluster

RainforestCluster Web Site

Other Useful Business Software

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Rate This Project

User Reviews

Be the first to post a review of RainforestCluster!

Additional Project Details

Operating Systems

Cygwin, Linux, Mac

Intended Audience

Information Technology, Science/Research

User Interface

Console/Terminal

Programming Language

Python

Related Categories

Python Systems Administration Software, Python Clustering Software, Python Distributed Computing Software

Registered

2014-03-09

Similar Business Software

AWS Batch

AWS Batch enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and...

See Software
EC2 Spot

Amazon EC2 Spot Instances let you take advantage of unused EC2 capacity in the AWS cloud. Spot Instances are available at up to a 90% discount compared to On-Demand prices. You can use Spot Instances for various stateless, fault-tolerant, or flexible applications such as big data, containerized...

See Software
AWS Elastic Fabric Adapter (EFA)

Elastic Fabric Adapter (EFA) is a network interface for Amazon EC2 instances that enables customers to run applications requiring high levels of inter-node communications at scale on AWS. Its custom-built operating system (OS) bypass hardware interface enhances the performance of inter-instance...

See Software
Elastigroup

Provision, manage and scale compute infrastructure on any cloud. Save up to 80% on your costs while ensuring SLA and high-availability. Elastigroup is a cluster software, designed to optimize performance and costs. It enables companies of all sizes and verticals to reliably leverage Cloud Excess...

See Software
Amazon Route 53

Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. It is designed to give developers and businesses an extremely reliable and cost-effective way to route end users to Internet applications by translating names into the numeric IP addresses like...

See Software
AWS CloudFormation

AWS CloudFormation is a infrastructure provisioning and management tool that provides you the ability to create resource templates that specifies a set of AWS resources to provision. The templates allow you to version control your infrastructure, and also easily replicate your infrastructure...

See Software