Search Results for "cluster" - Page 2
Sort By:
Showing 123 open source projects for "cluster"
View related business solutions-
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
Android Emulator Container Scripts
Minimal scripts to run the emulator in a container for various systems
...The project focuses on reproducibility and scale: you define which system image to boot, how to persist or reset data, and how many instances to run, then schedule them like any other workload. GPU acceleration, audio, and sensors can be enabled depending on your host and cluster capabilities, while fallbacks like SwiftShader keep things usable when no GPU is available. -
2
omegaml
MLOps simplified. From ML Pipeline ⇨ Data Product without the hassle
omega|ml is the innovative Python-native MLOps platform that provides a scalable development and runtime environment for your Data Products. Works from laptop to cloud. -
3
DualPipe
A bidirectional pipeline parallelism algorithm
...The main goal of DualPipe is to maximize overlap between computation and communication phases during distributed training, thus reducing idle GPU time (i.e. “pipeline bubbles”) and improving cluster efficiency. Traditional pipeline parallelism methods (e.g. 1F1B or staggered pipelining) leave gaps because forward and backward phases can’t fully overlap with communication. DualPipe addresses that by scheduling micro-batches from both ends of the pipeline in a bidirectional fashion—i.e. some micro-batches flow forward while others flow backward—so that computation on one partition can coincide with communication for another. -
4
Ansible for DevOps
Ansible for DevOps examples
Ansible for DevOps is a collection of Ansible playbooks, roles, and infrastructure-as-code examples that accompany the book Ansible for DevOps by Jeff Geerling. Rather than being theoretical, the examples span real-world infrastructure setups: multi-server orchestration, LAMP stacks, Docker deployments, Kubernetes cluster spins, rolling updates, and security hardening. You can clone the repo and play with actual scenarios using Vagrant, VirtualBox, or cloud hosts, making it ideal for both learning and reference in production readiness. The code is structured by chapter/topic, so you can pick a scenario (for example “nodejs deployment” or “ELK stack”) and dive into a fully featured Ansible solution rather than starting from scratch. ... -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
EKS Best Practices
A best practices guide for day 2 operations
...Because running Kubernetes in production demands many “day-2” considerations (upgrades, drift, monitoring, incident response), the guide provides practical advice beyond simple cluster provisioning. -
6
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. That design makes Matrix particularly well-suited for large-batch inference, model benchmarking, data curation, augmentation, or generation — whether for language, code, dialogue, or multimodal tasks. ... -
7
Diplomacy Cicero
Code for Cicero, an AI agent that plays the game of Diplomacy
...It supports two variants: Cicero (which handles full “press” negotiation) and Diplodocus (a variant focused on no-press diplomacy) as described in the README. The codebase is implemented primarily in Python with performance-critical components in C++ (via pybind11 bindings) and is configured to run in a high‐GPU cluster environment. Configuration is managed via protobuf files to define tasks such as self-play, benchmark agent comparisons, and RL training. The project is now archived and read-only, reflecting that it is no longer actively developed but remains publicly available for research use. -
8
Datahosting ipfs kubo-cluster
Managed IPFS Kubo pinning with IPFS Cluster replication
This project provides an open-source setup for IPFS Kubo (public and private pinning) combined with IPFS Cluster replication, designed for reliable, production-ready decentralized storage. It is suitable for developers, infrastructure operators, and Web3 projects that need predictable IPFS availability without maintaining complex node infrastructure. The system supports multiple retention plans for IPFS Kubo pinning, allowing users to choose how long data remains pinned, with transparent pay-per-GB prepaid pricing. ... -
9
WordCount
Count frequency of single, 2-word and 3-word clusters in a text
The program can read a text file and count the occurrences of single words and clusters of 2 and 3 words. The resulting list will be sorted in descending order (highest frequency on top). -
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
10
ASAM is a Ark Ascended dedicated server manager. Able to install and manage large clusters with different settings seamlessly. Built to fix what other launchers were lacking in control and simplicity.
-
11
Kaleidoscope-SDK
User toolkit for analyzing and interfacing with Large Language Models
kaleidoscope-sdk is a Python module used to interact with large language models hosted via the Kaleidoscope service available at: https://github.com/VectorInstitute/kaleidoscope. It provides a simple interface to launch LLMs on an HPC cluster, asking them to perform basic features like text generation, but also retrieve intermediate information from inside the model, such as log probabilities and activations. Users must authenticate using their Vector Institute cluster credentials. This can be done interactively instantiating a client object. This will generate an authentication token that will be used for all subsequent requests. ... -
12
GromacsProSuite
Graphical User Interface for Gromacs
This tool is an integrated graphical interface that simplifies molecular dynamics simulations using Gromacs. It provides a structured, tab-based environment to set up, execute, and analyze simulations data without complex command-line operations. The software automates tasks such as topology generation, solvation, ion addition, minimization, equilibration, and production runs while executing GROMACS commands in the background. Built-in monitoring tracks CPU, RAM, and disk usage to ensure... -
13
Genv
GPU environment management and cluster orchestration
Genv is an open-source environment and cluster management system for GPUs. Genv lets you easily control, configure, monitor and enforce the GPU resources that you are using in a GPU machine or cluster. It is intended to ease up the process of GPU allocation for data scientists without code changes. -
14
uvicorn-gunicorn-fastapi
Docker image with Uvicorn managed by Gunicorn
Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web applications in Python with performance auto-tuning. Optionally with Alpine Linux. -
15
Punica
Serving multiple LoRA finetuned LLM as one
...Punica introduces a serving architecture that allows multiple LoRA adapters to share the same base model during inference, significantly reducing memory consumption and computational overhead. The system includes specialized CUDA kernels that enable batched GPU operations across different LoRA models simultaneously. This design allows a single GPU cluster to host many task-specific models while maintaining high throughput and minimal latency. The architecture also includes scheduling mechanisms that coordinate requests from multiple tenants and distribute workloads efficiently across available resources. -
16
LLM Training Puzzles
What would you do with 1000 H100s
LLM Training Puzzles is a notebook-based collection of eight exercises about training large neural networks across many GPUs. It introduces the core primitives behind distributed language model training without requiring access to a thousand-machine cluster. The puzzles focus on memory efficiency, communication, partitioning, and compute pipelining. Learners work through simplified scenarios that expose the tradeoffs hidden inside large training systems. The hands-on format emphasizes deriving behavior rather than memorizing framework commands. Google Colab is recommended so readers can copy the notebook and begin experimenting quickly. ... -
17
KubiScan
A tool to scan Kubernetes cluster for risky permissions
A tool for scanning Kubernetes cluster for risky permissions in Kubernetes's Role-based access control (RBAC) authorization model. KubiScan helps cluster administrators identify permissions that attackers could potentially exploit to compromise the clusters. This can be especially helpful on large environments where there are lots of permissions that can be challenging to track. -
18
Spilo
Highly available elephant herd: HA PostgreSQL cluster using Docker
Spilo is a Docker-based HA PostgreSQL cluster built on Patroni and heavily optimized for Kubernetes environments. It includes components for failover, streaming replication, backups, and connection pooling. Spilo is used in production by Zalando and is designed to provide a resilient, self-healing Postgres cluster with minimal manual intervention. -
19
ClassyVision
An end-to-end PyTorch framework for image and video classification
...The library integrates seamlessly with PyTorch Hub for easy access to pretrained models and supports elastic training using PyTorch Elastic, making distributed training robust to changes in cluster resources or hardware failures. -
20
Scrapyd
A service daemon to run Scrapy spiders
Scrapyd can manage multiple projects and each project can have multiple versions uploaded, but only the latest one will be used for launching new spiders. A common (and useful) convention to use for the version name is the revision number of the version control tool you’re using to track your Scrapy project code. For example: r23. The versions are not compared alphabetically but using a smarter algorithm (the same packaging uses) so r10 compares greater to r9, for example. Scrapyd is an... -
21
Mars Framework
Mars is a tensor-based unified framework for large-scale data
...The project provides a tensor-based execution model that extends the capabilities of tools such as NumPy, pandas, and scikit-learn so that large datasets can be processed in parallel without rewriting code for distributed environments. Its architecture automatically divides large computational tasks into smaller chunks that can be executed across multiple nodes in a cluster, allowing complex analytics, machine learning workflows, and data transformations to run efficiently at scale. Mars is particularly useful for workloads that exceed the memory capacity of a single machine or require high levels of parallel processing. -
22
ceph-ansible
Ansible playbooks to deploy Ceph, the distributed filesystem
...After you’ve cloned the ceph-ansible repository, selected your branch and installed Ansible then you’ll need to create your inventory file, playbook and configuration for your Ceph cluster. The Ansible inventory file defines the hosts in your cluster and what roles each host plays in your Ceph cluster. The default location for an inventory file is /etc/ansible/hosts but this file can be placed anywhere and used with the -i flag of ansible-playbook. You must have a playbook to pass to the ansible-playbook command when deploying your cluster. ... -
23
dispy
Distributed and Parallel Computing with/for Python.
dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing. -
24
TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters
By combining salient features from the TensorFlow deep learning framework with Apache Spark and Apache Hadoop, TensorFlowOnSpark enables distributed deep learning on a cluster of GPU and CPU servers. It enables both distributed TensorFlow training and inferencing on Spark clusters, with a goal to minimize the amount of code changes required to run existing TensorFlow programs on a shared grid. -
25
Nameko
Python framework for building microservices
...Nameko includes an implementation of RPC over AMQP. It comprises the @rpc entry point, a proxy for services to talk to other services, and a standalone proxy that non-Nameko clients can use to make RPC calls to a cluster. The HTTP entry point is built on top of werkzeug, and supports all the standard HTTP methods (GET/POST/DELETE/PUT etc).
