Search Results for "cluster computing"
Sort By:
Showing 28 open source projects for "cluster computing"
View related business solutions-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
MicroK8s
Single-package Kubernetes for developers, IoT and edge
Low-ops, minimal production Kubernetes, for devs, cloud, clusters, workstations, Edge and IoT. MicroK8s automatically chooses the best nodes for the Kubernetes datastore. When you lose a cluster database node, another node is promoted. No admin needed for your bulletproof edge. MicroK8s is small, with sensible defaults that ‘just work’. A quick install, easy upgrades and great security make it perfect for micro clouds and edge computing. As the publishers of MicroK8s, we deliver the world’s most efficient multi-cloud, multi-arch Kubernetes. ... -
2
Covalent workflow
Pythonic tool for running machine-learning/high performance workflows
Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services. Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code. Covalent overcomes computational and operational challenges inherent in AI/ML experimentation. -
3
AWS ParallelCluster Node
Python package installed on the Amazon EC2 instances
aws-parallelcluster-node is the python package installed on the Amazon EC2 instances launched as part of AWS ParallelCluster. AWS ParallelCluster is an AWS-supported Open Source cluster management tool that makes it easy for you to deploy and manage High-Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. ... -
4
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. That design makes Matrix particularly well-suited for large-batch inference, model benchmarking, data curation, augmentation, or generation — whether for language, code, dialogue, or multimodal tasks. ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
5
Mars Framework
Mars is a tensor-based unified framework for large-scale data
Mars is a distributed computing framework designed to scale scientific computing and data science workloads across large clusters while preserving the familiar programming interfaces of common Python libraries. The project provides a tensor-based execution model that extends the capabilities of tools such as NumPy, pandas, and scikit-learn so that large datasets can be processed in parallel without rewriting code for distributed environments. -
6
dispy
Distributed and Parallel Computing with/for Python.
dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing. -
7
Nebula-Python-SDK
A python SDK for managing Nebula container orchestrator
...Nebula container orchestrator aims to help devs and ops treat IoT devices just like distributed Dockerized apps. It aim is to act as Docker orchestrator for IoT devices as well as for distributed services such as CDN or edge computing that can span thousands (possibly even millions) of devices worldwide and it does it all while being open-source and completely free. Nebula imposes no limits on the scale of the cluster, each component in it is designed to scale out to allow millions of workers to be managed by it. Designed to connect to devices that are spread around the globe Nebula is tolerant of network connection issues and will resync the device when it reconnects. ... -
8
Nebula docs
Documentation repo of nebula orchestration system
Nebula is a open source distributed Docker orchestrator designed for massive scales (tens of thousands of servers/worker devices), unlike Mesos/Swarm/Kubernetes it has the ability to have workers distributed on high latency connections (such as the internet) yet have the pods(containers) be managed centrally with changes taking affect (almost) immediately, this makes Nebula ideal for managing a vast cluster of servers\devices across the globe, some example use cases are appliances\virtual appliances located at clients data centers, edge computing, and POS systems. Ever wandered how your going to push an update to that smart fridge your company is working on as it's thousands of devices around the globe? wish you could have the assurance that your service will always use the latest code\envvars\etc in all of it's edge locations? ... -
9
Nebula worker
The worker node manager container which manages nebula nodes
Nebula is a open source distributed Docker orchestrator designed for massive scales (tens of thousands of servers/worker devices), unlike Mesos/Swarm/Kubernetes it has the ability to have workers distributed on high latency connections (such as the internet) yet have the pods(containers) be managed centrally with changes taking affect (almost) immediately, this makes Nebula ideal for managing a vast cluster of servers\devices across the globe, some example use cases are IoT devices, appliances\virtual appliances located at clients data centers, and edge computing. Nebula imposes no limits on the scale of the cluster, each component in it is designed to scale out to allow millions of workers to be managed by it. Designed to connect to devices that are spread around the globe Nebula is tolerant of network connection issues and will resync the device when it reconnects. ... -
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
10
Nebula reporter
The optional reporter container which reads nebula reports from Kafka
Nebula is an open source-distributed Docker orchestrator designed for massive scales (tens of thousands of servers/worker devices), unlike Mesos/Swarm/Kubernetes it has the ability to have workers distributed on high latency connections (such as the internet) yet have the pods(containers) be managed centrally with changes taking effect (almost) immediately, this makes Nebula ideal for managing a vast cluster of servers\devices across the globe. Ever wandered how your going to push an update... -
11
Python/xarray tutorial
Python/xarray tutorial for GEOS-Chem users
If the page is loaded successfully, you should see a Jupyter notebook interface. Then, click on the first notebook to get started. Jupyter combines Python code, execution results, plots, custom texts, and even Latex formulas in a single page. Besides using the Jupyter program, you can also view the static notebook on GitHub (e.g the first notebook). Python is free & open-source so can be easily installed on any machines. To best way to get the scientific Python environment is using the Conda... -
12
EvalAI
Evaluating state of the art in AI
...We allow the creation of an arbitrary number of evaluation phases and dataset splits, compatibility using any programming language, and organizing results in both public and private leaderboards. Certain large-scale challenges need special computing capabilities for evaluation. If the challenge needs extra computational power, challenge organizers can easily add their own cluster of worker nodes to process participant submissions while we take care of hosting the challenge, handling user submissions, and maintaining the leaderboard. EvalAI lets participants submit code for their agent in the form of docker images which are evaluated against test environments on the evaluation server. ... -
13
-
14
TaskManager
TaskManager manages calculation jobs in a computer cluster environment
TaskManager is an open source infrastructure software for distributing and managing calculation jobs in a Unix computer cluster environment. The TaskManager was designed to control the utilization of a set of hosts even if you are not the administrator of the system. The hosts are embedded in a Unix environment and the user's home directories are mounted on each host. The hosts may have different numbers of CPUs/cores and different kernels. Keep in mind that a user is able to log into each... -
15
RainforestCluster
Dynamically manage Amazon EC2 clusters
RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any... -
16
PBS Cluster Viz is a project to display information useful to admins and users about a computing cluster managed by a PBS-compatible resource manager. Information includes load and job distribution. Interactive as well as static output is available.
-
17
HPC cloud computing for OpenFOAM (R) users
-
18
SnowFlock enables high performance computing on virtual machine (VM)-based cloud environments. In SnowFlock, a VM transparently becomes a cluster of VMs by cloning in sub-second time into multiple copies executing on different physical hosts.
-
19
KestrelHPC is a software developed to make life easier to the people that want to have a Diskless Cluster. This Cluster is Multi-User and it has got many tools to configure your cluster. This software can be installed in a Debian or Ubuntu.
-
20
was moved to http://sourceforge.net/projects/cloudflu
-
21
This modular software package forms an integration point for Grid/Cloud/Cluster products. If provides several adapters which can easily be used to build complete Services/integrate products. It enables federation between multiple sites.
-
22
Python Integrated Parallel Programming EnviRonment (PIPPER), Python pre-parser that is designed to manage a pipeline, written in Python. It enables automated parallelization of loops. Think of it like OpenMP for Python, but it works in a computer cluster
-
23
LinuxTuples is an open-source implementation of David Gelernter's idea of a tuple space, designed to run on a networked cluster of Linux machines.
-
24
SSS-OSCAR is an integrated release of the SciDAC:Scalable System Software (SSS) suite using the OSCAR cluster installation framework. The SSS project seeks to provide a standard interface for effective management and utilization of terascale systems.
-
25
sharq is a batch-queueing system that allows for multiple users to make effective use of a beowulf cluster. sharq allows for prioritization based on a user's funding, and also based on the size of the job. sharq is very light-weight and easy to learn.
