Search Results for "compute cluster"
Sort By:
Showing 36 open source projects for "compute cluster"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
NVIDIA AI Cluster Runtime (AICR)
Tooling for optimized and reproducible GPU-accelerated AI runtime
NVIDIA AI Cluster Runtime (AICR) is an emerging project within NVIDIA’s AI infrastructure ecosystem focused on enabling advanced AI compute and runtime workflows, though publicly available documentation remains limited. Based on its positioning within NVIDIA’s repositories, it is designed to support scalable AI runtime environments, potentially addressing challenges related to orchestration, resource management, or reproducible AI execution. -
2
OpenResearch
Turn your coding agents into research agents
OpenResearch is a local-first workspace that turns coding agents into research agents capable of running structured scientific workflows. It works with Claude Code, Codex, OpenCode, and Cursor while letting each research direction use an independent session and Git worktree. Agents can review literature, propose hypotheses, modify code, launch experiments, inspect results, and generate research artifacts. Experiments are tracked through a Git-native tree that preserves lineage and immutable... -
3
AWS ParallelCluster Cookbook
The Chef cookbook used to build and bootstrap AWS ParallelCluster
AWS ParallelCluster is an AWS supported Open Source cluster management tool that makes it easy for you to deploy and manage High Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. -
4
future
R package: future: Unified Parallel and Distributed Processing in R
...It allows R expressions to be scheduled for future evaluation, with the result retrieved later, in a way decoupled from the specific backend used. This lets code be written in a way that works with sequential execution, multicore, multisession, cluster, or remote compute backends, without changing the high-level code. It handles automatic exporting of needed global variables/functions, managing of packages, RNG, etc. Automatic detection and export of global objects and functions needed by future expressions, so the user doesn’t need to manage that manually. Ability to control how futures are resolved. -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
AWS ParallelCluster Node
Python package installed on the Amazon EC2 instances
aws-parallelcluster-node is the python package installed on the Amazon EC2 instances launched as part of AWS ParallelCluster. AWS ParallelCluster is an AWS-supported Open Source cluster management tool that makes it easy for you to deploy and manage High-Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. ... -
6
Jupyter Enterprise Gateway
Enables Jupyter Notebooks to share resources across clusters
Jupyter Enterprise Gateway is a headless web server with a pluggable framework for anyone supporting multiple notebook users in a managed-cluster environment. Some of the core functionality it provides is better optimization of compute resources, improved multi-user support, and more granular security for your Jupyter notebook environment - making it suitable for enterprise, scientific, and academic implementations. From a technical perspective, Jupyter Enterprise Gateway is a web server that enables the ability to launch kernels on behalf of remote notebooks. ... -
7
omegaml
MLOps simplified. From ML Pipeline ⇨ Data Product without the hassle
omega|ml is the innovative Python-native MLOps platform that provides a scalable development and runtime environment for your Data Products. Works from laptop to cloud. -
8
Argo Workflows
Workflow engine for Kubernetes
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Define workflows where each step in the workflow is a container. Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG). Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using... -
9
Knative Serving
Kubernetes-based, scale-to-zero, request-driven compute
Knative Serving defines a set of objects as Kubernetes Custom Resource Definitions (CRDs). These resources are used to define and control how your serverless workload behaves on the cluster. The service.serving.knative.dev resource automatically manages the whole lifecycle of your workload. It controls the creation of other objects to ensure that your app has a route, a configuration, and a new revision for each update of the service. Service can be defined to always route traffic to the... -
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
10
PelicanHPC is an iso-hybrid (CD or USB) image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the image. The compute nodes boot by PXE, using the frontend node as the server. ...
-
11
GCMT
Gentoo Cluster Managment Tools
A collection of bash scripts to manage Gentoo System Images (GSIs), compute nodes, file servers and users on a Gentoo-based HPC cluster. GCMT works with PXE boot, dhcp, tftp, qmail, djbdns and tinydns, NIS, NFS, and TORQUE. -
12
LLM Training Puzzles
What would you do with 1000 H100s
LLM Training Puzzles is a notebook-based collection of eight exercises about training large neural networks across many GPUs. It introduces the core primitives behind distributed language model training without requiring access to a thousand-machine cluster. The puzzles focus on memory efficiency, communication, partitioning, and compute pipelining. Learners work through simplified scenarios that expose the tradeoffs hidden inside large training systems. The hands-on format emphasizes deriving behavior rather than memorizing framework commands. Google Colab is recommended so readers can copy the notebook and begin experimenting quickly. ... -
13
SuperEdge
An edge-native container management system for edge computing
SuperEdge is an open-source container management system for edge computing to manage compute resources and container applications in multiple edge regions. These resources and applications, in the current approach, are managed as one single Kubernetes cluster. A native Kubernetes cluster can be easily converted to a SuperEdge cluster. SuperEdge extends the powerful container orchestration and scheduling capabilities of Kubernetes to the edge. -
14
dispy
Distributed and Parallel Computing with/for Python.
dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing. -
15
Singularity
Open source container platform designed to be simple, fast, and secure
Singularity is an open-source container platform designed to be simple, fast, and secure. Many container platforms are available, but Singularity is designed for ease of use on shared systems and in high-performance computing (HPC) environments. -
16
progrep
Utility to show live progress, status & stats for running simulations
progrep is a command-line tool (Linux) to show live progress report, status & stats of a running simulation or compute job that executes a given number of iterations. It shows % completed, time remaining, time elapsed, number of threads, MPI_Rank(if any), CPU usage & speed (FPS). The FPS measures may be used in benchmarking, e.g. while optimizing HPC algorithms for performance. progrep supports both single-threaded and parallel (multicore/multinode - e.g. OpenMP/MPI) jobs. progrep can... -
17
Kubernetes The Hard Way
Bootstrap Kubernetes the hard way
Kubernetes The Hard Way is a hands-on guide to building a Kubernetes cluster from the ground up without using automation tools—no managed services, no scripts to hide the details. It walks you through every component: provisioning compute resources, generating TLS certificates, configuring etcd, bootstrapping the control plane, joining worker nodes, setting networking, and verifying everything works. The purpose is educational: by doing each step manually, you gain deep insight into how Kubernetes works under the hood—control plane components, kube-configs, networking, encryption, etc. ... -
18
CUDA-MEME
Ultrafast scalable motif discovery algorithm using GPU computing
mCUDA-MEME is a well-established ultrafast scalable motif discovery algorithm based on MEME (version 4.4.0) algorithm for multiple GPUs using a hybrid combination of CUDA, MPI and OpenMP parallel programming models. This algorithm is a further extension of CUDA-MEME (based on MEME version 3.5.4) with respect to accuracy and speed and has been tested on a GPU cluster with eight compute nodes and two Fermi-based Tesla S2050 (and Tesla-based Tesla S1070) quad-GPU computing systems, running the Linux OS with the MPICH2 library. The experimental results showed that our algorithm scales well with respect to both dataset sizes and the number of GPUs. At present, OOPS and ZOOPS models are supported, which are sufficient for most motif discovery applications. ... -
19
Manage node sets, node groups and execute commands on cluster nodes in parallel. Provides an event-based Python library to improve administration of large compute clusters or server farms. Command line tools: clush and nodeset included.
-
20
Cluster Bench is a framework to support a range of benchmarks and system tests for cluster systems. It automatically generates the required files for a batch system and can graphically display the results of completed benchmarks and other tests.
-
21
lustre-roll
lustre client roll for rocks clusters
This roll provides rpm packages to mount the lustre filesystem on the frontends and compute nodes of a rocks cluster. Please contact me if you need this roll for another kernel and/or lustre version. -
22
dna-bison
Bisulfite alignment On Nodes of a cluster
Bison allows users with access to a computer cluster to rapidly align whole-genome bisulfite sequencing or RRBS reads. It can align both directional and non-directional libraries and uses bowtie2. Multiple compute nodes are not absolutely required, but will make the alignment process faster. Further details available on the Wiki page. Help also available on SEQanswers (http://seqanswers.com/forums/showthread.php? -
23
Octree Method For Particles
simulation of particles using an oct-tree
...I realized that simulating a very large number of particles in e.g. electromagnetic sims can run into growing i/o demands. This is a way to overcome that hurdle. the low-level "octets" can be distributed among a cluster of computers, organized in a grid. they don't have to send each other their whole dataset - only a zoomed out image of it. the further away, the more zoomed out. thus, i/o traffic is greatly reduced. combine this with the fact that it's an O(N) algorithm, and what you get is an accurate particle-based physics simulator whose simulation speed scales with the number of particles per compute unit even for VERY large particle counts and VERY large computing arrays. ... -
24
ThunderstormDistributor
Distribute jobs to compute nodes on dynamic clusters
ThunderstormDistributor is a queuing system that distributes jobs and computational workload across dynamic clusters in the cloud. It manages the assignment of jobs to maximize CPU and memory usage and prevent oversubscription of compute nodes. It also performs advanced statistics collection on individual compute nodes and jobs to graph the distribution of disk, network, CPU, and memory usage over time, which facilitates the advanced optimization and tuning of computational workflows. -
25
RainforestCluster
Dynamically manage Amazon EC2 clusters
RainforestCluster is an Amazon EC2 python program that manages and load-balances dynamic clusters to allow for maximum workflow flexibility and speed at minimal cost. It enables one to quickly and cheaply create dynamic compute clusters in the cloud, which can then run computational pipelines generically. It is also able to optimize the use of spot instances - idle computers in Amazon's cloud that are available at drastically reduced cost (5x-10x cheaper) - but can be terminated at any moment if capacity drops or the bid price rises. It also provides pre-installed features such as GlusterFS distributed filesystems, ThunderstormDistributor queuing system, RAID 0 /scratch, password-less ssh, and automatic cluster management, for ease of use and maximum processing speed for computational tools. ...
