Search Results for "distributed file"
Sort By:
Showing 86 open source projects for "distributed file"
View related business solutions-
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
Tribler
Privacy enhanced BitTorrent client with P2P content discovery
...It introduces built-in anonymity using a Tor-like onion routing network and integrates its own blockchain for economic incentives and trust management. Tribler supports standard torrenting features along with distributed search, self-contained channels, and peer reputation. Its goal is to provide a fully autonomous file-sharing network without relying on external servers, search engines, or trackers. -
2
kokoro-onnx
TTS with kokoro and onnx runtime
...It focuses on running efficiently on commodity hardware, including macOS with Apple Silicon, while still delivering near real-time performance for many use cases. The project ships prebuilt model files and a simple example script, so you can go from installation to producing an audio.wav file in just a few steps. It supports multiple languages and voices, with a curated voice list and configuration via a VOICES file hosted alongside the models. The package is distributed on PyPI, meaning you can integrate it directly into applications or scripts using standard Python tooling. It also recommends pairing with an external G2P package to improve pronunciation quality, especially for more complex languages or names, and is licensed under permissive MIT and Apache-style licenses. -
3
Colossal-AI
Making large AI models cheaper, faster and more accessible
The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing. Together with better performance come larger model sizes. This imposes challenges to the memory wall of the current accelerator hardware such as GPU. It is never ideal to train large models such as Vision Transformer, BERT, and GPT on a single GPU or a single machine. There is an urgent demand to train models in a distributed environment.... -
4
huey
A little task queue for python
...It gives developers a clean API for running background jobs outside the main request or execution flow. The project supports several storage backends, including Redis, Valkey, Redict, SQLite, the file system, and in-memory storage. It can execute tasks with processes, threads, or greenlets, which makes it adaptable to different workloads. Huey also supports scheduled tasks, recurring tasks, retries, task priorities, result storage, expiration, locking, rate limits, timeouts, pipelines, groups, and chords. Overall, it is a practical option for Python teams that want background processing without the complexity of a heavier distributed task system. -
Paessler - Monitor Your Whole Network in MinutesWaiting weeks for a monitoring rollout isn't an option when infrastructure doesn't stop running. PRTG's auto-discovery scans your network and suggests from over 200 pre-configured sensor types, so you're watching servers, applications and devices within minutes, not after a multi-week deployment. Enterprise-strength monitoring, without the enterprise complexity. Start your free trial today.
-
5
ruff-pre-commit
A pre-commit hook for Ruff
...The tool supports both linting and formatting hooks, allowing developers to configure workflows that enforce style and correctness in a single step. It also provides flexibility in configuration, including selective file type targeting and optional automatic fixes through command-line arguments. -
6
Bottle
bottle.py is a fast and simple micro-framework for python applications
Bottle is a minimalist web framework for building small web applications and APIs in Python. It is distributed as a single file with no external dependencies, making it perfect for rapid development, prototyping, or embedded use. Despite its small size, Bottle supports routing, templates, request handling, and plugin support, offering a full-featured toolkit in an extremely compact package. -
7
Azure SDK for Python
Active development of the Azure SDK for Python
This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs or our versioned developer docs. For your convenience, each service has a separate set of libraries that you can choose to use instead of one, large Azure package. To get started with a specific library, see the README.md (or README.rst) file located in the library's project folder. Last stable versions of packages that have been provided for usage... -
8
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
Diskover Community Edition is an open source file system indexing and storage analytics platform designed to help organizations understand and manage large volumes of file data. It crawls file systems and indexes metadata using Elasticsearch, enabling fast search, analysis, and organization of files stored across different storage systems. It allows administrators and users to explore file structures, monitor storage usage, and gain insights into how data is distributed across infrastructure. ... -
9
ipwb
A distributed and persistent archive replay system using IPFS
InterPlanetary Wayback (ipwb) facilitates permanence and collaboration in web archives by disseminating the contents of WARC files into the IPFS network. IPFS is a peer-to-peer content-addressable file system that inherently allows deduplication and facilitates opt-in replication. ipwb splits the header and payload of WARC response records before disseminating into IPFS to leverage the deduplication, builds a CDXJ index with references to the IPFS hashes returned, and combines the header and... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
11
Cozette
A bitmap programming font optimized for coziness
A bitmap programming font optimized for coziness. Cozette is a 6x13px (bounding box; average 5px character width, 3px descent, 10px ascent, 8px cap height) bitmap font based on Dina, which itself is based on Proggy. It's also heavily inspired by Creep. I absolutely adore Creep, and was using it up until I got a higher-DPI screen for which it was slightly too small. That prompted me to make the bitmap font I always wished existed: Cozette; a small-but-not-tiny bitmap font with great coverage... -
12
OneFileLLM
Specify a github or local repo, github pull request
OneFileLLM is an open-source project designed to simplify the distribution and execution of large language model applications by packaging them into a single portable file. The concept behind the project is to eliminate the complexity normally associated with deploying AI systems, which often require multiple dependencies, frameworks, and configuration steps. Instead, the entire runtime environment, model interface, and application logic are bundled together into a single executable artifact. This design allows developers to share AI tools in a format that can be easily distributed and executed across different machines without complicated installation procedures. ... -
13
kubectl-aliases
Programmatically generated handy kubectl aliases
...This dramatically speeds up interactive Kubernetes workflows, especially when you spend a lot of time inspecting pods, services, deployments, and logs. The repository also includes a Python script used to generate the aliases, which advanced users can tweak if they want a custom alias set. It is distributed under the Apache-2.0 license and is widely used by Kubernetes practitioners. -
14
Ludwig AI
Low-code framework for building custom LLMs, neural networks
Declarative deep learning framework built for scale and efficiency. Ludwig is a low-code framework for building custom AI models like LLMs and other deep neural networks. Declarative YAML configuration file is all you need to train a state-of-the-art LLM on your data. Support for multi-task and multi-modality learning. Comprehensive config validation detects invalid parameter combinations and prevents runtime failures. Automatic batch size selection, distributed training (DDP, DeepSpeed), parameter efficient fine-tuning (PEFT), 4-bit quantization (QLoRA), and larger-than-memory datasets. ... -
15
Sandstorm
One API call, pull Claude agent, completely sandboxed
Sandstorm is an open-source project that wraps a powerful Claude-based AI agent within a completely sandboxed, ephemeral API service designed to make agentic AI workflows easy to deploy and scale without infrastructure complexity. The core idea is to provide “one API call” access to a robust Claude agent loop that runs inside a secure sandbox, so you can upload files, connect tools, and run long-running tasks — all managed behind a simple REST-style interface that disappears when the work is... -
16
Otter-Grader
A Python and R autograding solution
Otter Grader is a light-weight, modular open-source autograder developed by the Data Science Education Program at UC Berkeley. It is designed to work with classes at any scale by abstracting away the autograding internals in a way that is compatible with any instructor's assignment distribution and collection pipeline. Otter supports local grading through parallel Docker containers, grading using the autograder platforms of 3rd party learning management systems (LMSs), the deployment of an... -
17
UNICORE
UNiform Interface to COmputing and data REsources
UNICORE is a software suite for building federated systems, providing secure and seamless access to heterogeneous resource such as compute clusters and file systems. UNICORE deals with authentication, user mapping and authorization, and provides a comprehensive set of RESTful APIs for HPC access and workflows. Contributors: visit https://github.com/UNICORE-EU -
18
Video Frame Extractor
Extracts semi-random frames from all MP4 videos
... - Wait for the program to finish processing all videos. - Find your extracted frames in the 'extracted_frames' folder. The frames are extracted at evenly distributed points throughout each video. For example, if you choose 3 frames, they will be taken at the 25%, 50%, and 75% marks of each video. (Source code is included with the program .zip file.) -
19
<<Hack|Track GNU/Linux
Distro Penetrasing Live System Burn to USB Flash Disk & Run.
<<Hack|Track GNU/Linux is an open source operating system developed by the HTGL Project from Indonesia which provides penetration testing. -
20
migrid
A grid middleware with minimal user and resource requirements
[This project moved to Github and is no longer maintained here] Minimum intrusion Grid (MiG) is an attempt to design a new platform for Grid computing which is driven by a stand-alone approach to Grid, rather than integration with existing systems. The goal of the MiG project is to provide Grid infrastructure where the requirements on users and resources alike is as small as possible (minimum intrusion). MiG strives for minimum intrusion but will seek to provide a feature rich and... -
21
Osenpa Clipboard Manager
Search, organize and reuse clipboard history on Windows
...Current downloads are available from the Official Download link: https://osenpa.com/osenpa-clipboard-manager. Earlier EXE files have been removed from this SourceForge page. License terms accompanying previously distributed copies are unchanged. -
22
DAE Tools Project
Object-oriented equation-based modelling and optimisation software
...It is not a modelling language nor a collection of numerical libraries but rather a higher level structure – an architectural design of interdependent software components providing an API for: - Model development/specification - Activities on developed models, such as simulation, optimisation, sensitivity analysis and parameter estimation - Processing of the results, such as plotting and exporting to various file formats - Report generation - Code generation, co-simulation and model exchange The following class of problems can be solved by DAE Tools: - Initial value problems of implicit form - Index-1 DAE systems - With lumped or distributed parameters - Steady-state or dynamic - Continuous with some elements of event-driven systems -
23
CiteFlow
Read research PDFs and organize source-linked notes and citations
Current official release: 1.2.1. Windows 10 1809+ or Windows 11 (x64). CiteFlow is a free, closed-source Windows workspace for research PDFs you already have. Read sources side by side, highlight passages, keep page-linked notes and preserve citation context. Search exact wording locally, or install an optional model for beta semantic suggestions. Optional local OCR supports scanned sources. Prepare APA, MLA, Chicago and IEEE citation drafts, then check metadata, quotations and... -
24
build nanoGPT
Video+code lecture on building nanoGPT from scratch
Build nanoGPT is an educational reproduction of GPT-style language-model training built step by step from an initially empty file. Its Git history is intentionally organized so learners can follow each architectural and training improvement as it is introduced. The accompanying video lecture explains how the code develops into a reproduction of the 124-million-parameter GPT-2 model. The project covers tokenization, transformer architecture, optimization, distributed training, data loading, and performance improvements. ... -
25
100 Days of Code YouTube
Source code for 100 days of code python course on YouTube
...Early material covers variables, data types, input, strings, conditions, loops, functions, and common collections. Intermediate lessons explore exceptions, modules, file I/O, functional programming, and extensive object-oriented programming concepts. Later days introduce command-line utilities, generators, regular expressions, AsyncIO, multithreading, and multiprocessing. Exercises and solution days are distributed throughout the sequence so learners can repeatedly apply what they have studied.
