Search Results for "distributed storage"
Sort By:
Showing 46 open source projects for "distributed storage"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
1
Zipkin
Distributed tracing system to gather timing data
Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data. If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether or not operations... -
2
Jaeger
Monitor and troubleshoot transactions in complex distributed systems
...It is used for monitoring and troubleshooting microservices-based distributed systems. OpenTracing compatible data model and instrumentation libraries include Go, Java, Node, Python, C++ and C#. Jaeger uses consistent upfront sampling with individual per service/endpoint probabilities and it has multiple storage backends: Cassandra, Elasticsearch, memory. -
3
Harvester
Open source hyperconverged infrastructure (HCI) software
...It is an open-source alternative designed for operators seeking a cloud-native HCI solution. Harvester runs on bare metal servers and provides integrated virtualization and distributed storage capabilities. In addition to traditional virtual machines (VMs), Harvester supports containerized environments automatically through integration with Rancher. It offers a solution that unifies legacy virtualized infrastructure while enabling the adoption of containers from core to edge locations. Harvester is an enterprise-ready, easy-to-use infrastructure platform that leverages local, direct attached storage instead of complex external SANs. ... -
4
Foundatio
Pluggable foundation blocks for building distributed apps
Pluggable foundation blocks for building loosely coupled distributed apps. Includes implementations in Redis, Azure, AWS, RabbitMQ and in memory (for development). When building several big cloud applications we found a lack of great solutions (that's not to say there aren't solutions out there) for many key pieces to building scalable distributed applications while keeping the development experience simple. Wanted to build against abstract interfaces so that we could easily change... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
5
Bacalhau
Community-driven, simple, yet powerful framework
Bacalhau is a decentralized compute platform for running jobs on data stored across distributed networks, like IPFS or Filecoin, without moving the data to centralized cloud environments. It allows developers to run containerized workloads close to where the data lives, reducing latency, cost, and privacy risks. Bacalhau supports various runtime environments and is designed to make decentralized data processing as accessible as traditional cloud computing. It’s especially useful for... -
6
CubeFS
cloud-native file store
...Support replicas and erasure coding engines, users can choose flexibly according to business scenarios. Easy to build a PB or EB-scale distributed storage service, and each module can be expanded horizontally. Supports multi-tenant management and provides fine-grained tenant isolation policies. Supports multi-level caching, and multiple high-performance replication protocols, and optimizes specific performance for small files. -
7
Apache Drill
Apache Drill is a distributed MPP query layer for self describing data
Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel. Get faster insights without the overhead (data loading, schema creation and maintenance, transformations, etc.) Analyze the multi-structured and nested data in non-relational datastores directly without transforming or restricting the data. -
8
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
9
Datahike
A durable Datalog implementation adaptable for distribution
Datahike is a durable Datalog database powered by an efficient Datalog query engine. This project started as a port of DataScript to the hitchhiker-tree. All DataScript tests are passing, but we are still working on the internals. Having said this we consider Datahike usable for medium sized projects, since DataScript is very mature and deployed in many applications and the hitchhiker-tree implementation is heavily tested through generative testing. We are building on the two projects and... -
Powerful App Monitoring Without Surprise BillsTired of monitoring tools that punish you for scaling? AppSignal offers transparent, predictable pricing with every feature unlocked on every plan. Track errors, monitor performance, detect anomalies, and manage logs across Ruby, Python, Node.js, and more. Trusted by developers since 2012 with free dev-to-dev support. No credit card required to start your 30-day trial.
-
10
Micro Cloud
A distributed OS built for the Cloud
Micro addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provides a set of services which act as the building blocks of a platform. Micro deals with the complexity of distributed systems and provides simpler programmable abstractions to build on. Micro is the all encompassing end to end platform experience from source to running and beyond built with a developer first focus. Micro’s goal is to abstract away the... -
11
JuiceFS
JuiceFS is a distributed POSIX file system built on top of Redis
...Purposely built to serve big data scenarios such as self-driving model training, recommendation engine, and Next-generation Gene Sequencing, JuiceFS specializes in high performance and easier management of tens of billion of files management. We bring JuiceFS to developers with the hope that it will be easy to use, reliable, high-performance, and solve all your file storage problems in a cloud environment. -
12
MinIO Client Quickstart Guide
Fast tool to manage MinIO clusters
MinIO Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3-compatible cloud storage service (AWS Signature v2 and v4). MinIO is distributed under GNU AGPLv3. See the source tree for complete information on all components of the work and their licenses. MinIO is a high-performance object storage solution that provides an Amazon Web Services S3-compatible API and supports all core S3 features. MinIO is built to deploy anywhere - public or private cloud, bare metal infrastructure, orchestrated environments, and edge infrastructure. ... -
13
Zeebe
Distributed Workflow Engine for Microservices Orchestration
Automate processes at scale with unprecedented performance and resilience. Zeebe is the workflow and decision engine that powers Camunda Platform 8. Zeebe’s cloud-native design provides the performance, resilience, and security enterprises need to future-proof their process orchestration efforts. Zeebe distributes data across all brokers in a cluster with storage directly on the server filesystem. If one broker goes down, another can replace it with no data loss. This pre-configured... -
14
Apache Spark
A unified analytics engine for large-scale data processing
...The built-in MLlib library provides scalable machine learning algorithms, while GraphX enables graph computations integrated with data pipelines. Spark supports multiple languages—Scala, Java, Python, R—and connects with many storage systems like HDFS, S3, Cassandra, and streaming platforms like Kafka, making it a versatile choice for big data workloads in analytics, ETL, and data science. -
15
KubeSphere
The container platform tailored for Kubernetes multi-cloud, datacenter
KubeSphere is a distributed operating system for cloud-native application management, using Kubernetes as its kernel. It provides a plug-and-play architecture, allowing third-party applications to be seamlessly integrated into its ecosystem. KubeSphere is also a multi-tenant container platform with full-stack automated IT operation and streamlined DevOps workflows. It provides developer-friendly wizard web UI, helping enterprises to build out a more robust and feature-rich platform, which... -
16
Redpanda
Redpanda is a streaming data platform for developers
...We are building a real-time streaming engine for modern applications - from the enterprise to the solo dev prototyping a react application on her laptop. We go beyond the Kafka protocol, into the future of streaming with inline WASM transforms and geo-replicated hierarchical storage. A new platform that scales with you from the smallest projects to petabytes of data distributed across the globe. We provide a very simple build system that uses your system libraries. We recommend users leverage our pre-built stable releases which are vetted, tested, and reproducible with exact versions of the entire transitive dependency graph, including exact compilers all built from source. ... -
17
Gorse Recommender System Engine
An open source recommender system service written in Go
An open-source recommender system service written in Go. Recommend items from Popular, latest, user-based, item-based and collaborative filtering. Search the best recommendation model automatically in the background. Support horizontal scaling in the recommendation stage after single node training. Support Redis, MySQL, Postgres, MongoDB, and ClickHouse as its storage backend. Expose RESTful APIs for data CRUD and recommendation requests. Analyze online recommendation performance from... -
18
Spring Cloud Config Server
External configuration (server and client) for Spring Cloud
Spring Cloud Config provides server-side and client-side support for externalized configuration in a distributed system. With the Config Server, you have a central place to manage external properties for applications across all environments. The concepts on both client and server map identically to the Spring Environment and PropertySource abstractions, so they fit very well with Spring applications but can be used with any application running in any language. -
19
System Design
Learn how to design systems and prepare for system design interviews
This project is an open, course-style repository designed to help you learn system design from fundamentals through advanced, interview-ready thinking. It organizes core networking and distributed-systems concepts into a structured path, so you can build intuition before jumping into “design X” exercises. It covers the building blocks that show up in real architectures, such as DNS, load balancing, caching, CDNs, proxies, scalability and availability tradeoffs, and storage patterns, then connects those pieces to larger system goals. ... -
20
Deequ
Deequ is a library built on top of Apache Spark
Deequ is a library built atop Apache Spark that enables defining “unit tests for data” — that is, formal constraints or checks on datasets to ensure data quality along dimensions such as completeness, uniqueness, value ranges, correlations, etc. It can scale to large datasets (billions of rows) by translating those data checks into Spark jobs. Deequ supports advanced features like a metrics repository for storing computed statistics over time, anomaly detection of data quality metrics, and... -
21
Tyk API Gateway
Open Source API Gateway written in Go
...Built from the ground up to be the fastest API gateway on the planet. It does not depend on a legacy proxy underneath. It has no 3rd party dependencies aside from Redis for distributed rate-limiting and token storage. Tyk Gateway can also be deployed as part of a larger Full Lifecycle API Management platform Tyk Self-Managed which also includes Management Control Plane, Dashboard GUI and Developer Portal. -
22
Emitter
High performance, distributed, low latency publish-subscribe platform
Emitter is a real-time communication service for connecting online devices. The Publish-Subscribe messaging API is built for speed and security. The simple and efficient messaging platform for all of your gaming, IoT and web apps. Emitter can be used for online gaming and mobile apps by satisfying the requirements for low latency, binary messaging, and high throughput. It can also be used for real-time web applications such as dashboards or visual analytics or chat systems. Moreover, Emitter... -
23
Xinorbis (and tools)
Source code for Xinorbis and associated tools
Xinorbis is a very powerful hard disk, folder and storage analyser. It uses a sophisticated mix of graphs, tables, and tree displays to give a complete, and unprecedented, picture of the contents of any hard disk, SSD, folder, removable or network drive. In fact, if your PC can see it, Xinorbis can scan it. Xinorbis makes it easy to see the contents, structure, file distribution, and file composition of any attached storage device. -
24
Atomix
A Kubernetes toolkit for building distributed applications
Atomix is a cloud-native runtime for building stateful, scalable, configurable, and reliable distributed applications in Kubernetes. The Atomix API provides a set of high-level building blocks (referred to as distributed primitives) for building distributed systems. The architecture of Atomix incorporates the lessons learned from experience over the past decade of building high-availability cloud infrastructure. The primary focus of the project is to decouple applications from specific data... -
25
Aurae
Distributed systems runtime daemon written in Rust
Aurae is on a mission to be the most loved and effective way of managing workloads on a node. Our hope is that by bringing a better set of controls to a node, we can unlock brilliant higher-order distributed systems in the future. Aurae deploys a memory-safe 1 runtime daemon, process manager, and PID-1 initialization system to remotely schedule processes, containers, and virtual machines as well as set node configurations (e.g., networking storage). Through system proportioning and enterprise workload isolation techniques, the Aurae open-source project can complement higher-order schedulers and control planes (such as Kubernetes) as Aurae supports the usage of multi-tenant workloads and enterprise identities all the way down to the socket layer.
