Search Results for "cluster computing"
Sort By:
Showing 151 open source projects for "cluster computing"
View related business solutions-
Auth0 for AI Agents now in GAConnect your AI agents to apps and data more securely, give users control over the actions AI agents can perform and the data they can access, and enable human confirmation for critical agent actions.
-
Grafana: The open and composable observability platformGrafana is the open source analytics & monitoring solution for every database.
-
1
Apache Sedona
Cluster computing framework for processing large-scale geospatial data
Apache Sedona™ is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark and Apache Flink, with a set of out-of-the-box distributed Spatial Datasets and Spatial SQL that efficiently load, process, and analyze large-scale spatial data across machines. According to our benchmark and third-party research papers, Sedona runs 2X - 10X faster than other Spark-based geospatial data systems on computation-intensive query workloads. ... -
2
Kraken
P2P Docker registry capable of distributing TBs of data in seconds
Kraken is a P2P-powered Docker registry that focuses on scalability and availability. It is designed for Docker image management, replication, and distribution in a hybrid cloud environment. With pluggable backend support, Kraken can easily integrate into existing Docker registry setups as the distribution layer. Kraken has been in production at Uber since early 2018. In our busiest cluster, Kraken distributes more than 1 million blobs per day, including 100k 1G+ blobs. At its peak... -
3
Dagger.jl
A framework for out-of-core and parallel execution
Dagger.jl is a framework for out-of-core and parallel computing in Julia that allows users to construct and execute dynamic task graphs. It is designed for large-scale, distributed, and memory-efficient computations. Dagger supports lazy evaluation and scheduling across multiple threads or machines, enabling high-performance workflows for data processing, scientific computing, and machine learning. -
4
kubeinvaders
Gamified Chaos Engineering Tool for Kubernetes
This project is part of the landscape of Cloud Native Computing Foundation in the Observability and Analysis - Chaos Engineering section. Backed by the teams at platform engineering.it and devopstribe.it, which provides enterprise-grade features and certified resilience services for your Kubernetes infrastructure. -
Respond 100x faster, more accurately, and improve your documentationSlash response times for questionnaires, audits, and RFPs by up to 90%. OptiValue.ai automates the heavy lifting, freeing your team to focus on strategic priorities with intuitive tools for seamless review and validation.
-
5
Kubernetes Handbook
Cloud native application architecture practice handbook
...Kubernetes is a container orchestration and scheduling engine developed by Google in June 2014 based on its internal Borg system. Google contributed it as an initial and core project to the CNCF (Cloud Native Computing Foundation). In recent years, it has gradually developed a cloud native Ecology. The goal of Kubernetes is to provide a specification to describe the architecture of the cluster. -
6
LINKERD
Ultralight, security-first service mesh for Kubernetes
Enterprise power without enterprise complexity. Linkerd adds security, observability, and reliability to any Kubernetes cluster. 100% open source, CNCF graduated, and written in Rust. Instantly add latency-aware load balancing, request retries, timeouts, and blue-green deploys to keep your applications resilient. Incredibly small and blazing fast Linkerd2-proxy micro-proxy written in Rust for security and performance. Self-contained control plane, incrementally deployable data plane, and... -
7
MicroK8s
Single-package Kubernetes for developers, IoT and edge
Low-ops, minimal production Kubernetes, for devs, cloud, clusters, workstations, Edge and IoT. MicroK8s automatically chooses the best nodes for the Kubernetes datastore. When you lose a cluster database node, another node is promoted. No admin needed for your bulletproof edge. MicroK8s is small, with sensible defaults that ‘just work’. A quick install, easy upgrades and great security make it perfect for micro clouds and edge computing. As the publishers of MicroK8s, we deliver the world’s most efficient multi-cloud, multi-arch Kubernetes. ... -
8
sparklyr
R interface for Apache Spark
sparklyr is an R package that provides seamless interfacing with Apache Spark clusters—either local or remote—while letting users write code in familiar R paradigms. It supplies a dplyr-compatible backend, Spark machine learning pipelines, SQL integration, and I/O utilities to manipulate and analyze large datasets distributed across cluster environments. -
9
Redis
An in-memory database that persists on disk
...Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. You can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing an element to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set. To achieve top performance, Redis works with an in-memory dataset. Depending on your use case, you can persist your data either by periodically dumping the dataset to disk or by appending each command to a disk-based log. -
Software for managing apps and accounts | WebCatalogTurn websites into desktop apps with WebCatalog Desktop—your all-in-one tool to manage apps and accounts. Switch between multiple accounts, organize apps by workflow, and access a curated catalog of desktop apps for Mac and Windows.
-
10
tsuru
Open source and extensible Platform as a Service (PaaS)
Tsuru is an open source Platform as a Service software, focused in Developer productivity. Tsuru goes beyond 12 factor apps. Run any application written in any language or framework. The deploy process is really simple with just one command. Grow your application dynamically allocating resources with ease. Let your developers code and understand the business instead of solving infrastructure problems or handling large configuration files. We built the platform as a service atop of... -
11
PelicanHPC is an iso-hybrid (CD or USB) image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the image.
-
12
AlphaZero.jl
A generic, simple and fast implementation of Deepmind's AlphaZero
Beyond its much publicized success in attaining superhuman level at games such as Chess and Go, DeepMind's AlphaZero algorithm illustrates a more general methodology of combining learning and search to explore large combinatorial spaces effectively. We believe that this methodology can have exciting applications in many different research areas. Because AlphaZero is resource-hungry, successful open-source implementations (such as Leela Zero) are written in low-level languages (such as C++)... -
13
AWS ParallelCluster Cookbook
The Chef cookbook used to build and bootstrap AWS ParallelCluster
AWS ParallelCluster is an AWS supported Open Source cluster management tool that makes it easy for you to deploy and manage High Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. -
14
Covalent workflow
Pythonic tool for running machine-learning/high performance workflows
Covalent is a Pythonic workflow tool for computational scientists, AI/ML software engineers, and anyone who needs to run experiments on limited or expensive computing resources including quantum computers, HPC clusters, GPU arrays, and cloud services. Covalent enables a researcher to run computation tasks on an advanced hardware platform – such as a quantum computer or serverless HPC cluster – using a single line of code. Covalent overcomes computational and operational challenges inherent in AI/ML experimentation. -
15
Smile
Statistical machine intelligence and learning engine
Smile is a fast and comprehensive machine learning engine. With advanced data structures and algorithms, Smile delivers the state-of-art performance. Compared to this third-party benchmark, Smile outperforms R, Python, Spark, H2O, xgboost significantly. Smile is a couple of times faster than the closest competitor. The memory usage is also very efficient. If we can train advanced machine learning models on a PC, why buy a cluster? Write applications quickly in Java, Scala, or any JVM... -
16
AWS ParallelCluster Node
Python package installed on the Amazon EC2 instances
aws-parallelcluster-node is the python package installed on the Amazon EC2 instances launched as part of AWS ParallelCluster. AWS ParallelCluster is an AWS-supported Open Source cluster management tool that makes it easy for you to deploy and manage High-Performance Computing (HPC) clusters in the AWS cloud. Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS. It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch and Slurm. ... -
17
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. That design makes Matrix particularly well-suited for large-batch inference, model benchmarking, data curation, augmentation, or generation — whether for language, code, dialogue, or multimodal tasks. ... -
18
Apache Seata
High-performance, open source distributed transaction solution
Seata is a distributed transaction solution for microservices that provides consistent, cross-service commits without forcing every team to adopt the same persistence model. Its architecture separates responsibilities into a global coordinator and per-service participants, so business services remain decoupled while transactions are orchestrated centrally. Multiple modes are supported—AT (automatic, SQL-based with undo logs), TCC (try-confirm-cancel), Saga (long-running compensation), and... -
19
Java Multiprocessing
Java explicit multiprocessing and cluster management tool
▪ JMP library allows explicit multiprocessing in Java with API that resembles Java Multithreading which is an essential built-in Java feature. JMP allows running a piece of Java code on a separate process either locally or on remote machine. ▪ If the code in question is not present on the target machine, it will be dynamically uploaded. ▪ JMP allows setting affinity for a new process explicitly instead of relying on the OS. ▪ JMP also contains a simple cluster management / load balancing... -
20
Bufferstack.IO Super Container OS
Live OS running Docker Container Engine, Kubernetes, Xen Hypervisor
Super Container OS makes use of os level virtualization to image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings, all this by making use of Docker Container Engine and systemd-nspawn Also included in is Xen Hypervisor and Kubernetes to create full fledged container cluster After installing this OS on HDD, please use Portainer to manage your Docker images. Portainer 2.0.0 is now preinstalled along with Helm (tool for managing Charts. Charts are packages of pre-configured Kubernetes resources) Run portainer as root and use your browser to log in portainer running on port number 9000 Refer details of Helm here - https://github.com/helm/helm Super Container OS is based on Bufferstack,IO Computing Platform, for more details, please check https://bufferstack.io -
21
Amplicon_Sequencing_Worfklow
Analyzing amplicon data from sequences to stats
...All you need is to have the latest version of R installed, some experience with the command line & shell, and enough memory to run all of the programs. There are also instructions provided in case you are running these analyses via a computing cluster/Slurm workload manager. You can choose to go through the workflow using either an Rmd script, an html file, or a PDF, or via the homepage link provided. If you have questions or concerns, please don't hesitate to reach out. Thanks! -
22
KubeOperator
An open source, lightweight Kubernetes distribution
...Compared with heavyweight PaaS platforms such as OpenShift, KubeOperator only focuses on solving one problem, which is to help enterprises plan (Day 0), deploy (Day 1), and operate (Day 2) production-level K8S clusters, and achieve the ultimate. Supports multiple computing, storage and networking scenarios. Integrate Ansible and Terraform. Support online environment and offline environment deployment. Provides a visual web UI. Supports cluster planning, deployment and operations. Easily run workloads like machine learning, high-performance computing, and more. Quickly deploy and manage applications in K8S. ... -
23
SuperEdge
An edge-native container management system for edge computing
SuperEdge is an open-source container management system for edge computing to manage compute resources and container applications in multiple edge regions. These resources and applications, in the current approach, are managed as one single Kubernetes cluster. A native Kubernetes cluster can be easily converted to a SuperEdge cluster. SuperEdge extends the powerful container orchestration and scheduling capabilities of Kubernetes to the edge. -
24
dispy
Distributed and Parallel Computing with/for Python.
dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing. -
25
Erda
An enterprise-grade Cloud-Native application platform for Kubernetes
Erda is an open-source platform created by Terminus to ensure the development of microservice applications. It provides DevOps, microservice governance, and multi-cloud management capabilities. The multi-cloud architecture based on Kubernetes and application-centric DevOps and microservice governance can make the development, operation, monitoring, and problem diagnosis of complex business applications simpler and more efficient. erda is the core repository that implements all the RESTful...
