Search Results for "distributed computing"
Sort By:
Showing 1535 open source projects for "distributed computing"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Endpoint Management with Remote Control Built InSave 50% on your first year when you switch from your existing RMM to Splashtop AEM. Automate patching, track inventory, and jump directly into 1-click unattended remote sessions without switching tools.
-
1
Distributed System Series
Educational repository containing technical notes
DistributedSystem-Notes is an open-source educational repository containing technical notes and practical references about distributed computing and infrastructure architecture. It explores the principles behind systems that distribute processing, storage, and communication across multiple machines. The material covers network architecture, operating systems, distributed computing, data storage, virtualization, and cloud infrastructure. Readers can study concepts such as resource scheduling, data consistency, distributed communication, and infrastructure management. ... -
2
NumPy
The fundamental package for scientific computing with Python
Fast and versatile, the NumPy vectorization, indexing, and broadcasting concepts are the de-facto standards of array computing today. NumPy offers comprehensive mathematical functions, random number generators, linear algebra routines, Fourier transforms, and more. NumPy supports a wide range of hardware and computing platforms, and plays well with distributed, GPU, and sparse array libraries. The core of NumPy is well-optimized C code. Enjoy the flexibility of Python with the speed of compiled code. ... -
3
Zipkin
Distributed tracing system to gather timing data
Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data. If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether or not operations... -
4
Dagger.jl
A framework for out-of-core and parallel execution
Dagger.jl is a framework for out-of-core and parallel computing in Julia that allows users to construct and execute dynamic task graphs. It is designed for large-scale, distributed, and memory-efficient computations. Dagger supports lazy evaluation and scheduling across multiple threads or machines, enabling high-performance workflows for data processing, scientific computing, and machine learning. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
PowerJob
Enterprise job scheduling middleware with distributed computing
...Four execution modes are supported, including stand-alone, broadcast, Map and MapReduce. Distributed computing resources could be utilized in MapReduce mode, try the magic out here! Both job dependency management and data communications between jobs are supported. Developers can write their processors in Java, Shell, Python, and will subsequently support multilingual scheduling via HTTP. -
6
BOINC
Open-source software for volunteer computing and grid computing
BOINC (Berkeley Open Infrastructure for Network Computing) is an open-source platform that enables distributed computing using volunteered computer resources. It allows researchers to harness massive amounts of processing power from public participants for scientific projects such as climate research, disease modeling, and astrophysics. BOINC supports cross-platform deployment and is backed by a large, active community. -
7
PolarDB-X
PolarDB-X is a cloud native distributed SQL Database
PolarDB-X is a cloud-native distributed SQL database designed to handle high concurrency, massive storage, and complex querying scenarios. It features a shared-nothing architecture that decouples computing from storage, providing scalability and flexibility for various applications. -
8
JupyterLab
JupyterLab computational environment
JupyterLab is the next-generation web-based user interface for Project Jupyter. Try it on Binder. JupyterLab follows the Jupyter Community Guides. JupyterLab enables you to work with documents and activities such as Jupyter notebooks, text editors, terminals, and custom components in a flexible, integrated, and extensible manner. You can arrange multiple documents and activities side by side in the work area using tabs and splitters. Documents and activities integrate with each other,... -
9
Parallax
Parallax is a distributed model serving framework
Parallax is a decentralized inference framework designed to run large language models across distributed computing resources. Instead of relying on centralized GPU clusters in data centers, the system allows multiple heterogeneous machines to collaborate in serving AI inference workloads. Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
fugue
A unified interface for distributed computing
Fugue is a unified interface for distributed computing that lets users execute Python, Pandas, and SQL code on Spark, Dask, and Ray with minimal rewrites. -
11
ShardingSphere
Distributed database ecosphere
Apache ShardingSphere is an open-source ecosystem consisted of a set of distributed database solutions, including 3 independent products, JDBC, Proxy & Sidecar (Planning). They all provide functions of data scale out, distributed transaction and distributed governance, applicable in a variety of situations such as Java isomorphism, heterogeneous language and cloud native. Apache ShardingSphere aiming at reasonably making full use of the computation and storage capacity of existed database in... -
12
Kubeflow Trainer
Distributed AI Model Training and LLM Fine-Tuning on Kubernetes
...The platform supports a wide range of machine learning frameworks, including PyTorch, JAX, Hugging Face, DeepSpeed, and XGBoost, making it highly flexible for different AI use cases. One of its key innovations is the integration of MPI-based distributed computing within Kubernetes, allowing efficient communication between nodes for high-performance training. It also includes advanced scheduling capabilities through integrations with tools like Kueue and Volcano, enabling topology-aware resource allocation and multi-cluster job orchestration. -
13
Antseed
An open market for AI inference. No gatekeepers
Antseed is a free, open-source peer-to-peer network for discovering, providing, and accessing AI inference services without a centralized gateway. It connects AI service providers with consumers through a distributed hash table and encrypted peer-to-peer communications. Providers can offer services backed by local Ollama models, Anthropic, OpenAI-compatible APIs, or specialized AI workflows. Buyers discover available providers and route requests according to pricing, latency, and reputation... -
14
Dask
Parallel computing with task scheduling
Dask is a Python library for parallel and distributed computing, designed to scale analytics workloads from single machines to large clusters. It integrates with familiar tools like NumPy, Pandas, and scikit-learn while enabling execution across cores or nodes with minimal code changes. Dask excels at handling large datasets that don’t fit into memory and is widely used in data science, machine learning, and big data pipelines. -
15
Pholcus
Distributed high-concurrency crawler software written in pure golang
Pholcus is a high-concurrency crawler software written in pure Go language that supports distributed, only used for programming learning and research. It supports three operating modes of stand-alone, server and client, and has three operating interfaces, Web, GUI, and command line; simple and flexible rules, concurrent batch tasks, and rich output methods (mysql/mongodb/kafka/csv/excel, etc.); In addition, it also supports horizontal and vertical grabbing modes, and a series of advanced... -
16
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
17
CUDA Python
Performance meets Productivity
...It integrates tightly with the broader Python GPU ecosystem, including Numba for kernel compilation and CCCL for parallel primitives, allowing developers to write performant code without leaving Python. The toolkit also includes utilities for profiling, memory management, distributed computing, and numerical operations, making it suitable for scientific computing, AI, and data processing workloads. -
18
Infinispan
Infinispan is an open source data grid platform
Infinispan is a distributed in-memory data grid and caching system designed for high-performance computing. It allows applications to scale dynamically by distributing data across multiple nodes, reducing latency and improving resilience. -
19
EdgeChains
EdgeChains.js is Full-Stack GenAI library
EdgeChains.js is a full-stack generative AI library that provides front-end, back-end, APIs, prompt management, and distributed computing capabilities, with core prompts and chains managed declaratively in Jsonnet. At EdgeChains, we take a unique approach to Generative AI - we think Generative AI is a deployment and configuration management challenge rather than a UI and library design pattern challenge. We build on top of a tech that has solved this problem in a different domain - Kubernetes Config Management - and bring that to Generative AI. ... -
20
Kraken
P2P Docker registry capable of distributing TBs of data in seconds
Kraken is a P2P-powered Docker registry that focuses on scalability and availability. It is designed for Docker image management, replication, and distribution in a hybrid cloud environment. With pluggable backend support, Kraken can easily integrate into existing Docker registry setups as the distribution layer. Kraken has been in production at Uber since early 2018. In our busiest cluster, Kraken distributes more than 1 million blobs per day, including 100k 1G+ blobs. At its peak... -
21
sparklyr
R interface for Apache Spark
sparklyr is an R package that provides seamless interfacing with Apache Spark clusters—either local or remote—while letting users write code in familiar R paradigms. It supplies a dplyr-compatible backend, Spark machine learning pipelines, SQL integration, and I/O utilities to manipulate and analyze large datasets distributed across cluster environments. -
22
Apache Kyuubi
Apache Kyuubi is a distributed and multi-tenant gateway
...At the server-side, Kyuubi server and engines' multi-tenant architecture provides the administrators a way to achieve computing resource isolation, data security, high availability, high client concurrency, etc. -
23
Ubicloud
Open source alternative to AWS. Elastic compute, block storage
Ubicloud is an open-source cloud platform that aims to provide a decentralized alternative to traditional hyperscale cloud providers. It focuses on building a federated network of providers where individuals and organizations can contribute infrastructure and offer compute, storage, and networking resources. Ubicloud emphasizes transparency and openness: APIs, orchestration, and management layers are open, enabling users to audit and customize their infrastructure instead of relying on... -
24
Datahike
A durable Datalog implementation adaptable for distribution
Datahike is a durable Datalog database powered by an efficient Datalog query engine. This project started as a port of DataScript to the hitchhiker-tree. All DataScript tests are passing, but we are still working on the internals. Having said this we consider Datahike usable for medium sized projects, since DataScript is very mature and deployed in many applications and the hitchhiker-tree implementation is heavily tested through generative testing. We are building on the two projects and... -
25
Xtuner
A Next-Generation Training Engine Built for Ultra-Large MoE Models
Xtuner is a large-scale training engine designed for efficient training and fine-tuning of modern large language models, particularly mixture-of-experts architectures. The framework focuses on enabling scalable training for extremely large models while maintaining efficiency across distributed computing environments. Unlike traditional 3D parallel training strategies, XTuner introduces optimized parallelism techniques that simplify scaling and reduce system complexity when training massive models. The engine supports training models with hundreds of billions of parameters and enables long-context training with sequence lengths reaching tens of thousands of tokens. ...
