RainforestCluster

RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. Originally it was developed as a different version for the Wall Lab at Harvard CBMI.

Features

Create Amazon EC2 custom clusters dynamically
Add, remove, and loadbalance nodes
Create large shared gluster volumes with spanning across multiple masters
Distribute jobs efficiently using ThunderstormDistributor
View machine, job, and cluster statistics on a web-based interface

Project Activity

See All Activity >

License

MIT License

Follow RainforestCluster

RainforestCluster Web Site

Other Useful Business Software

Forever Free Full-Stack Observability | Grafana Cloud

Our generous forever free tier includes the full platform, including the AI Assistant, for 3 users with 10k metrics, 50GB logs, and 50GB traces.

Built on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.

Create free account

Rate This Project

User Reviews

Be the first to post a review of RainforestCluster!

Additional Project Details

Operating Systems

Cygwin, Linux, Mac

Intended Audience

Information Technology, Science/Research

User Interface

Console/Terminal

Programming Language

Python

Related Categories

Python Systems Administration Software, Python Clustering Software, Python Distributed Computing Software

Registered

2014-03-09

Similar Business Software

Google Compute Engine

Compute Engine is Google's infrastructure as a service (IaaS) platform for organizations to create and run cloud-based virtual machines. Computing infrastructure in predefined or custom machine sizes to accelerate your cloud transformation. General purpose (E2, N1, N2, N2D) machines provide a...

See Software
Comet Backup

Start running backups and restores in less than 15 minutes! Fast, secure backup software for businesses and IT providers. Comet is a flexible, all-in-one backup platform available in 13 languages. You choose your backup destination, server location, configuration and setup. Backup to your...

See Software
Files.com

Files.com is a cloud-native Managed File Transfer (MFT) platform that unifies file transfers, sharing, and automation across any cloud, protocol, or partner. It connects 50+ storage systems — including Amazon S3, Azure, Google Drive, SharePoint, Dropbox, and Box — presenting them as a single...

See Software
Google Cloud Platform

Google Cloud is a cloud-based service that allows you to create anything from simple websites to complex applications for businesses of all sizes. New customers get $300 in free credits to run, test, and deploy workloads. All customers can use 25+ products for free, up to monthly usage...

See Software
Kasm Workspaces

Kasm Workspaces streams your workplace environment directly to your web browser…on any device and from any location. Kasm uses our high-performance streaming and secure isolation technology to provide web-native Desktop as a Service (DaaS), application streaming, and secure/private web...

See Software
ScalaHosting

• Simple website management with our own SPanel control panel - a free cPanel/WHM alternative • Free and effortless migration - we will migrate all your websites and mailboxes at no additional cost and without any downtime • Unconditional anytime money-back guarantee - just to show you how...

See Software