Open Source Python Clustering Software

Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. Supports clusters up to 2000 nodes in size.

Genv is an open-source environment and cluster management system for GPUs. Genv lets you easily control, configure, monitor and enforce the GPU resources that you are using in a GPU machine or cluster. It is intended to ease up the process of GPU allocation for data scientists without code changes.

Open HPI is an open source implementation of the SA Forum's Hardware Platform Interface (HPI). HPI provides an abstracted interface to managing computer hardware, typically for chassis and rack based servers. The new location for the project is https://github.com/open-hpi/openhpi

Can be deployed on AWS, GCE, Azure, OpenStack, vSphere, Equinix Metal (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal. Highly available cluster. Composable (Choice of the network plugin for instance). Supports most popular Linux distributions. Continuous integration tests. The list of available docker versions is 18.09, 19.03, and 20.10. The recommended docker version is 20.10. The kubelet might break on docker's non-standard version numbering (it no longer uses semantic versioning). To ensure auto-updates don't break your cluster look into e.g. yum version lock plugin or apt pin). The target servers must have access to the Internet in order to pull docker images. Otherwise, additional configuration is required. The target servers are configured to allow IPv4 forwarding. If using IPv6 for pods and services, the target servers are configured to allow IPv6 forwarding.

UPDATE: Chromium is no longer updated or maintained. The project is frozen. Chromium is a flexible framework for scalable real-time rendering on clusters of workstations, derived from the Stanford WireGL project code base.

GGI stands for "General Graphics Interface", and it is a project that aims to develop a reliable, stable and fast graphics system that works everywhere. We want to allow any program using GGI to run on any platform requiring at most a recompile.

dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing.

A Console-based BitTorrent Client with built-in scheduler for handling multiple sessions. It is designed to manage sessions in queue easily without heavy-weight GUI. External module can search for new torrents in trackers and submit it automatically.

This lets you used the PVM (parallel virtual machine) API from within Python.

Slurm is a replacement for other resource management software and schedulers like gridengine or torque. The slurm roll integrates very well into a rocks clusters installation. In the folder addons, there are a lot of useful rolls for rocks clusters 6.1 and 6.2. These rolls do not depend on slurm. Please see Discussion for further information and support

Condor GUI is a tool for managing and working with one or more HTCondor pools. It was developed at the Otto von Guericke University Magdeburg, Germany, to support the daily work using HTCondor. Version History: Version 0.9.3: - QDarkStyleSheet is an optional package now Version 0.9.2: - New layout of job file editor - Keyword arguments in job file editor - Linux compatibility

PBS Cluster Viz is a project to display information useful to admins and users about a computing cluster managed by a PBS-compatible resource manager. Information includes load and job distribution. Interactive as well as static output is available.

RPyC, or Remote Python Call, is a transparent and symmetrical python library for remote procedure calls, clustering and distributed-computing.

asyncoro is a Python framework for developing concurrent, distributed, network programs with asynchronous completions and coroutines. Asynchronous completions implemented in asyncoro are sockets (non-blocking sockets), database cursors, sleep timers and locking primitives. Programs developed with asyncoro have same logic and structure as Python programs with threads, except for a few syntactic changes. asyncoro supports socket I/O notification mechanisms epoll, kqueue, /dev/poll (and poll and select, where necessary), and Windows I/O Completion Ports (IOCP) for high performance and scalability, and SSL for security

coNCePTuaL is a toolset for rapidly generating portable, readable, and reproducible network-performance tests. coNCePTuaL can perform the equivalent of many pages of C code with just a few mouse clicks or lines of code in a domain-specific language.

aws-parallelcluster-node is the python package installed on the Amazon EC2 instances launched as part of AWS ParallelCluster. AWS ParallelCluster is an AWS-supported Open Source cluster management tool that makes it easy for you to deploy and manage High-Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. AWS ParallelCluster facilitates both quick start proof of concepts (POCs) and production deployments. You can build higher-level workflows, such as a Genomics portal that automates the entire DNA sequencing workflow, on top of AWS ParallelCluster.

ActiveGrid is an Enterprise Web 2.0 solution that allows the composition of code-free applications that comply with corporate IT standards. Technologies include Python, Java, XForm, Xpath, WSDL, CSS, XML Schema (XSD), XACML, and BPEL.

This is a system for scheduling the tasks in a heterogenous environment such as grid system .It will be a effective and extensible system.

The Arusha Project seeks to provide a framework and/or tools for collaborative system administration of multi-platform Unix sites with many dozens of machines.

Banshell is a tool to scriptably execute commands on a remote machine. It was designed with the goal of increasing security by only allowing a specified set of commands to be executed.

Bluesfield: An experimental distributed computing platform for the support of grassroots movements.

Broke-Aid is a cheap SAN alternative with built-in Brocade(R)-type fencing mechanism. It uses GNU/Linux and open source software components. Broke-Aid has been tested with Red Hat Enteprise Linux (versions 3 and 4) Cluster Suite and Global File System.

CIMHA is aimed to create integrated suite to provision and manage HA clusters in virtual environment. It analyzes kernel/hardware health of cluster nodes by using VESPER(linux kernel prober) and migrates the Resources based on probe signals.

Clubmask is a resource and job manager that ties together the Bproc global process space system, SIS, the System Installation Suite for installing Linux clusters, the Maui HPC Scheduler, and Supermon resource managers. The goal of Clubmask is to provide

This project allow to test network availability on multiple nodes. It is built as a plugin architecture in order to use it on multiples networks. Test is based on three steps : module status, local visibility of network, and interconnection between nodes