Search Results for "distributed storage"
Sort By:
Showing 24 open source projects for "distributed storage"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across infrastructure. ... -
2
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
3
Databend
Cloud-native open source data warehouse for analytics and AI queries
...Built in Rust, the system focuses on high performance, scalability, and efficient data processing for analytical queries. It is designed with a separation of compute and storage, allowing compute nodes to scale independently while storing data in object storage systems. This architecture enables cost-efficient storage and elastic scaling for workloads that involve large datasets and complex queries. Databend provides a unified engine capable of handling analytics, vector search, and full-text search within a single platform. ... -
4
Flan Scan
A pretty sweet vulnerability scanner
Flan Scan is a lightweight open-source network vulnerability scanner designed to make it easy to detect exposed services, open ports, and associated vulnerabilities across IP ranges or network segments as part of security audit and compliance workflows. It is essentially a thin wrapper around the widely-used Nmap scanner, augmenting it with scripts and tooling that transform raw Nmap output into vulnerability-focused reports that map detected services to known CVEs, making results more... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
migrid
A grid middleware with minimal user and resource requirements
[This project moved to Github and is no longer maintained here] Minimum intrusion Grid (MiG) is an attempt to design a new platform for Grid computing which is driven by a stand-alone approach to Grid, rather than integration with existing systems. The goal of the MiG project is to provide Grid infrastructure where the requirements on users and resources alike is as small as possible (minimum intrusion). MiG strives for minimum intrusion but will seek to provide a feature rich and... -
6
Datahosting ipfs kubo-cluster
Managed IPFS Kubo pinning with IPFS Cluster replication
...Retention options include short-term and long-term storage, making it suitable for websites, datasets, backups, and distributed applications. For higher availability, IPFS Cluster replication is available with 1, 2, or 3 replicas, enabling redundancy across nodes and improved resilience against node failure. Bandwidth usage is tracked separately, with included allowances and predictable overage pricing.https://github.com/branislav1989/ipfs-kubo-private-public-ipfs-cluste -
7
CostPal
CostPal is your personal finance manager
...It's purpose is to act as a scratchpad, not a standalone application for managing personal finance - for that you have CostPal. Currently only Dropbox is supported as a cloud storage used for data exchange. -
8
BerryNet
Deep learning gateway on Raspberry Pi and other edge devices
This project turns edge devices such as Raspberry Pi into an intelligent gateway with deep learning running on it. No internet connection is required, everything is done locally on the edge device itself. Further, multiple edge devices can create a distributed AIoT network. At DT42, we believe that bringing deep learning to edge devices is the trend towards the future. It not only saves costs of data transmission and storage but also makes devices able to respond according to the events shown in the images or videos without connecting to the cloud. One of the applications of this intelligent gateway is to use the camera to monitor the place you care about. ... -
9
ECDICT
Free English to Chinese Dictionary Database
ECDICT is a comprehensive English–Chinese dictionary dataset packaged for developers who need an offline, queryable lexicon for applications, NLP, or educational tools. It aggregates headwords, phonetics, parts of speech, translations, and example information into formats that are easy to integrate. The project provides multiple distribution forms—commonly SQLite/CSV/StarDict-style files—so you can choose the right storage and query approach for your app. Because it’s offline and local, it... -
Stop Storing Third-Party Tokens in Your DatabaseRolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
-
10
haipproxy
Distributed proxy IP pool for web crawlers using Scrapy and Redis
HAipproxy is a distributed proxy IP pool system designed to collect, manage, and provide large numbers of proxy addresses for web crawling tasks. It automatically crawls proxy resources from the internet and aggregates them into a centralized pool that can be accessed by distributed spiders and scraping systems. It is built using Python and relies on Scrapy for high-performance crawling while Redis is used for data storage, communication, and task coordination between components. ... -
11
Hierarchical cluster engine HCE
Hierarchical Cluster Engine project
The main idea of this project – to implement the solution that can be used to: construct custom network mesh or distributed network cluster structure with several relations types between nodes, formalize the data flow processing goes from upper node level central source point to down nodes and backward, formalize the management requests handling from multiple source points, support native reducing of multiple nodes results (aggregation, duplicates elimination, sorting and so on), internally support powerful full-text search engine and data storage, provide transactions-less and transactional requests processing, support flexible run-time changes of cluster infrastructure, have many languages bindings for client-side integration APIs in one product build on C++ language... ... -
12
RedMQ
Simple, flexible messaging with Redis.
RedMQ is a Redis-based messaging system that is specifically designed to support flexible messaging schemes, exposing an API that allows messages to be filtered by a number of message attributes. At its core is a Python TCP server built on Asynchronous Messaging Protocol and Twisted's AMP module. This means RedMQ can support clients in any language, though only two clients - PHP and Python - have been implemented to date. The API is designed with flexibility in mind, providing native support... -
13
Kadna is a small curses program written in Python that creates a secure and synchronizable database. This is intended for distributed password and small file (e.g. cryptographic keys) storage. This project in not longer maintained.
-
14
This is attempt to make an open source bookmarking system that supports tagging, distributed data storage, genetic "splicing" of strains of bookmarking tags and much more!
-
15
-
16
Universal information crawler is a fast precise and reliable Internet crawler. Uicrawler is a program/automated script which browses the World Wide Web in a methodical, automated manner and creates the index of documents that it accesses.
-
17
iROS is a meta-operating system for technology-rich "interactive rooms". The core components (Event Heap, DataHeap, iCrafter) provide communication, data storage, and service management for an iRoom.
-
18
-
19
rpmdiff is a utility to create a binary patch from one version of an rpm to another. This binary patch can then be distributed to end-users of a distribution who have installed the first version of the rpm and they will be upgraded to the second version.
-
20
Sfooi - distributed serverless p2p/file storage operating over protocol similiar (but not identicall) to airhook, using DHT as routing medium.
-
21
LemFS is a distributed, redundant data storage system, designed to utilize unused disk space on networked work stations and desktop PCs.
-
22
This software is a distributed replicated blob server (inspired by the google file system paper http://www.cs.rochester.edu/sosp2003/papers/p125-ghemawat.pdf ). It stores your blobs ( <=> files) on a given number of your servers.
-
23
Zeus Grid is a Grid Computing environment usefull to run systems in heterogenous machines at same time. In this first step, it will only compile, run and collect application results and file storage.
-
24
- Previous
- You're on page 1
- Next
