Search Results for "cluster computing"
Sort By:
71 projects for "cluster computing" with 1 filter applied:
-
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Apache Sedona
Cluster computing framework for processing large-scale geospatial data
Apache Sedona™ is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark and Apache Flink, with a set of out-of-the-box distributed Spatial Datasets and Spatial SQL that efficiently load, process, and analyze large-scale spatial data across machines. According to our benchmark and third-party research papers, Sedona runs 2X - 10X faster than other Spark-based geospatial data systems on computation-intensive query workloads. ... -
2
Redis
An in-memory database that persists on disk
...Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. You can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing an element to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set. To achieve top performance, Redis works with an in-memory dataset. Depending on your use case, you can persist your data either by periodically dumping the dataset to disk or by appending each command to a disk-based log. -
3
Apache Seata
High-performance, open source distributed transaction solution
Seata is a distributed transaction solution for microservices that provides consistent, cross-service commits without forcing every team to adopt the same persistence model. Its architecture separates responsibilities into a global coordinator and per-service participants, so business services remain decoupled while transactions are orchestrated centrally. Multiple modes are supported—AT (automatic, SQL-based with undo logs), TCC (try-confirm-cancel), Saga (long-running compensation), and... -
4
Kubeflow Trainer
Distributed AI Model Training and LLM Fine-Tuning on Kubernetes
...The platform supports a wide range of machine learning frameworks, including PyTorch, JAX, Hugging Face, DeepSpeed, and XGBoost, making it highly flexible for different AI use cases. One of its key innovations is the integration of MPI-based distributed computing within Kubernetes, allowing efficient communication between nodes for high-performance training. It also includes advanced scheduling capabilities through integrations with tools like Kueue and Volcano, enabling topology-aware resource allocation and multi-cluster job orchestration. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. That design makes Matrix particularly well-suited for large-batch inference, model benchmarking, data curation, augmentation, or generation — whether for language, code, dialogue, or multimodal tasks. ... -
6
Mars Framework
Mars is a tensor-based unified framework for large-scale data
Mars is a distributed computing framework designed to scale scientific computing and data science workloads across large clusters while preserving the familiar programming interfaces of common Python libraries. The project provides a tensor-based execution model that extends the capabilities of tools such as NumPy, pandas, and scikit-learn so that large datasets can be processed in parallel without rewriting code for distributed environments. -
7
dispy
Distributed and Parallel Computing with/for Python.
dispy is a generic and comprehensive, yet easy to use framework for creating and using compute clusters to execute computations in parallel across multiple processors in a single machine (SMP), among many machines in a cluster, grid or cloud. dispy is well suited for data parallel (SIMD) paradigm where a computation (Python function or standalone program) is evaluated with different (large) datasets independently. dispy supports public / private / hybrid cloud computing, fog / edge computing. -
8
EECluster
Tool for energy-efficient resource management in HPC clusters
...This software has been developed by Quantum and High Performance Computing group (QHPC). See https://qhpc.uniovi.es/ -
9
Python/xarray tutorial
Python/xarray tutorial for GEOS-Chem users
If the page is loaded successfully, you should see a Jupyter notebook interface. Then, click on the first notebook to get started. Jupyter combines Python code, execution results, plots, custom texts, and even Latex formulas in a single page. Besides using the Jupyter program, you can also view the static notebook on GitHub (e.g the first notebook). Python is free & open-source so can be easily installed on any machines. To best way to get the scientific Python environment is using the Conda... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
SAFMQ: Store and Forward Message Queue, message oriented middleware. Uses include Assured Async Messaging, SOA, Delayed/Batch, and Cluster/Grid Computing. The SAFMQ server provides cross platform communication among C++, PHP, Java, and .NET clients.
-
11
A framework to run MATLAB programs as batch jobs. Features a structured input description, integrity constraints and GUI.Independent parts of a job can execute in parallel on a cluster computer. Developed at Freiburg Brain Imaging (FBI) - http://fbi.uniklinik-freiburg.de/
-
12
DMTK
Microsoft Distributed Machine Learning Toolkit
The Microsoft Distributed Machine Learning Toolkit (DMTK) is an open-source framework created to support scalable machine learning across distributed computing environments. Developed by Microsoft Research, the toolkit provides infrastructure and algorithms designed to train large models efficiently on clusters of machines rather than a single system. At its core is a parameter-server architecture called Multiverso, which manages model parameters and synchronizes updates across distributed... -
13
Postgres-XL is a PostgreSQL-based scale-out cluster that handles both OLTP write intensive workloads as well as OLAP/BI type of workloads thanks to MPP parallelism. Postgres-XL is fully ACID and supports cluster-wide Multi-version Concurrency Control (MVCC) for global consistency and performance. There are commercial companies such as 2ndQuadrant, who provide support and consulting around Postgres-XL. Please visit http://2ndquadrant.com/en/resources/postgres-xl/ for more details.
-
14
Moscrack is a perl application designed to facilitate cracking WPA keys on a cluster of computers. This is accomplished with any one of Mosix clustering software, ssh, rsh or Pyrit connectivity to nodes. Cluster nodes can run any Un*x variant (includ
-
15
Summingbird
Streaming MapReduce with Scalding and Storm
Summingbird is a streaming + batch hybrid computation framework developed by Twitter. Its aim is to let developers express data aggregation pipelines in a unified way, where the same logic can run either in real time (stream) or in batch mode, and the results can be merged or reconciled. In effect, Summingbird abstracts over multiple execution engines (such as Storm, Scalding, etc.) to provide one high-level program that composes transformations and aggregations, and then executes them in... -
16
-
17
OpenSearchServer Store
A file server with a RESTFul web service.
An open source (Apache 2 license) file server software with a RESTFul web service for file management in a cluster of nodes. Including 4 set of API - Store API: Upload, download files and directories - Process API: Managing the long term process runs - Glacier API: Uploading from the server to Glacier - Cluster API: Manage the nodes in the cluster -
18
neuranep
Neural Network Engineering Platform
A parallel-programming framework for concurrently running large numbers of small autonomous jobs, or microthreads, across multiple cores in a CPU or CPUs in a cluster. NeuraNEP emulates a distributed processing environment capable of handling millions of microthreads in parallel, for example running neural networks with millions of spiking cells. Microthreads are general processing elements that can also represent non-neural elements, such as cell populations, extracellular space, emulating... -
19
MediaEncodingCluster is an Enterprise Class, Video Cluster Environment with a Plattform Independent Client - Server Architecture for distributed video/audio converting/encoding tool over a grid Computing Network Design. more on http://docs.codergrid.de
-
20
PBS Cluster Viz is a project to display information useful to admins and users about a computing cluster managed by a PBS-compatible resource manager. Information includes load and job distribution. Interactive as well as static output is available.
-
21
HADOS
High Availability Distributed Object Storage
HADOS is a server software for storing files in a cluster of server. The goal is to handle high availability by storing copy of the same file on several nodes. HADOS provides RESTFUL APIs to easily store, check or retrieve files. Using the powerful cluster APIs you can retrieve the files whatever the node that hosts it. To avoid any single point of failure, it is possible to apply a request to any node of the cluster, there is no master node. -
22
EGNLA
Heterogeneous cluster for acceleration of linear algebra computations
This project deals with a new concept of the heterogeneous grid for acceleration of the numerical linear algebra computing. We design this grid with respect to maximal ratio between cost and computational power. It allows a parallelization of scientific codes with minimal programming effort. We also optimize grid concept to be less sensitive to network parameters. -
23
HPC cloud computing for OpenFOAM (R) users
-
24
Equalizer is the standard middleware to create parallel OpenGL-based applications. Please visit https://github.com/Eyescale for current development information.
-
25
Simdist lets you harness the power of cluster computing without any knowledge of parallel libraries such as MPI, and with no restrictions on programming language. Primarily targeted at evolutionary computing and similar master-slave configurations.
