Search Results for "throughput" - Page 6
Sort By:
Showing 502 open source projects for "throughput"
View related business solutions-
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Light-4J
A fast, lightweight and more productive microservices framework
...In 2014, I realized that the IT industry was moving from Monolithic to Microservices and from on-premise data centers to the public clouds. To reduce the production cost for my applications, I need to find a lightweight platform that has a small memory footprint, high throughput, and low latency in Java. Java EE and Spring/Spring Boot are too heavy to be considered. Other lightweight Java platforms all have different issues on the separation between the business logic and technical cross-cutting concerns. Moreover, they are not cloud-native or designed for container/cloud. -
2
Apache Flink
Stream processing framework with powerful stream
Apache Flink is a distributed engine for stateful computations over data streams and batches, designed for low-latency processing at scale. Its core runtime executes dataflow graphs with fine-grained backpressure and checkpointing, allowing applications to recover consistently from failures. Flink’s event-time model and watermarks enable accurate out-of-order processing, windowing, and complex time semantics that typical real-time systems struggle with. Developers program against high-level... -
3
Grafbase
The Grafbase GraphQL Federation Gateway
...By supporting GraphQL federation standards, the platform enables teams to combine multiple subgraphs or services into a unified schema that clients can query as a single endpoint. Grafbase is designed for large-scale production environments, emphasizing high throughput, low latency, and efficient resource usage through its Rust implementation. The system also includes tools for schema governance, API lifecycle management, and collaboration workflows, helping engineering teams safely evolve complex APIs over time. -
4
trench
Open-Source Analytics Infrastructure
Trench is an open-source analytics infrastructure designed for tracking events and performing real-time analysis of application data at scale. The system is built on top of high-performance data technologies including Apache Kafka and ClickHouse, which allows it to ingest and process very large volumes of events while maintaining fast query performance. It was originally developed to solve scaling challenges in product analytics systems where traditional relational databases become... -
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
5
FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs
...The platform enables developers to deploy trained models quickly using optimized inference pipelines that support GPUs, specialized AI accelerators, and other hardware architectures. FastDeploy includes advanced acceleration technologies such as speculative decoding, multi-token prediction, and efficient KV cache management to improve throughput and latency during inference. It also offers compatibility with OpenAI-style APIs and vLLM-like interfaces, allowing developers to integrate deployed models easily into existing applications and services. -
6
slime LLM
slime is an LLM post-training framework for RL Scaling
slime is an open-source large language model (LLM) post-training framework developed to support reinforcement learning (RL)-based scaling and high-performance training workflows for advanced LLMs, blending training and rollout modules into an extensible system. It offers a flexible architecture that connects high-throughput training (e.g., via Megatron-LM) with a customizable data generation pipeline, enabling researchers and engineers to iterate on new RL training paradigms effectively. The framework is designed to support a wide range of training modes, allowing both synchronous and asynchronous RL workflows and programmable rollout interfaces that simplify experimentation with custom environments and reward signals. ... -
7
cloudflare-speed-cli
CLI for internet speed test via cloudflare
cloudflare-speed-cli is a Rust-based command-line internet speed testing tool that interfaces with Cloudflare’s public speed test endpoints, letting users measure download and upload throughput, latency, and more from a terminal. It presents results in an interactive text-user-interface (TUI) that charts real-time metrics as tests run, making it easy to visually inspect performance trends without leaving the console. The tool also stores historical test results and can export measured data as structured JSON for scripting, logging, or integration with automation tools. ... -
8
OCRBase
MD/.JSON Document OCR and structured data extraction API
...Instead of treating OCR as a one-off script, it presents an API-driven workflow where documents are submitted as jobs and processed through a queue-based architecture that can handle high throughput. The core output is designed for downstream automation, producing structured results like JSON according to user-defined schemas while also providing readable formats like Markdown for human review or indexing. It includes real-time job progress updates via WebSockets, which makes it easier to integrate into UIs, dashboards, or ingestion systems where users need feedback on long-running document processing. -
9
OpenVINO Model Server
A scalable inference server for models optimized with OpenVINO
...It’s implemented in C++ for scalability and efficiency, making it suitable for both edge and cloud deployments where inference workloads must be reliable and high throughput. The server exposes model inference via standard network protocols like REST and gRPC, allowing any client that speaks those protocols to request predictions remotely, abstracting away the complexity of where and how the model runs. It supports model deployment in diverse environments including Docker, bare-metal machines, and Kubernetes clusters, and is especially useful in microservices architectures where AI services need to scale independently. ... -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
10
nanochat
The best ChatGPT that $100 can buy
nanochat is a from-scratch, end-to-end “mini ChatGPT” that shows the entire path from raw text to a chatty web app in one small, dependency-lean codebase. The repository stitches together every stage of the lifecycle: tokenizer training, pretraining a Transformer on a large web corpus, mid-training on dialogue and multiple-choice tasks, supervised fine-tuning, optional reinforcement learning for alignment, and finally efficient inference with caching. Its north star is approachability and... -
11
ZeusDB Vector Database
Blazing-fast vector DB with similarity search and metadata filtering
ZeusDB is a vector database built for fast, scalable similarity search with strong production ergonomics. It combines high-performance approximate nearest neighbor indexes with clean APIs and metadata filtering so applications can retrieve semantically relevant items at low latency. The storage layer is designed for durability and growth, supporting sharding, replication, and background compaction while keeping query tails predictable. Developers get multiple ingestion paths—batch,... -
12
TensorStore
Library for reading and writing large multi-dimensional arrays
...Transactional semantics allow atomic updates and consistent snapshots, which is essential for large, shared datasets used by ML and scientific workflows. The library is engineered for scalability—background caching, chunk sharding, and retryable operations keep throughput high even over unreliable networks. With language bindings, it fits into Python-heavy analysis pipelines while retaining a fast C++ core. -
13
Honggfuzz
Security oriented software fuzzer
honggfuzz is a general-purpose, high-performance fuzzer that mixes coverage feedback with practical crash triage to uncover memory-safety and logic bugs. It supports multiple fuzzing modes—stdin, file, and networking—so targets can be exercised the same way they run in production. Instrumentation via compiler hooks or hardware/perf counters guides mutations toward previously unseen edges, while persistent mode keeps the target process alive to amortize startup costs. The tool integrates... -
14
fairseq2
FAIR Sequence Modeling Toolkit 2
fairseq2 is a modern, modular sequence modeling framework developed by Meta AI Research as a complete redesign of the original fairseq library. Built from the ground up for scalability, composability, and research flexibility, fairseq2 supports a broad range of language, speech, and multimodal content generation tasks, including instruction fine-tuning, reinforcement learning from human feedback (RLHF), and large-scale multilingual modeling. Unlike the original fairseq—which evolved into a... -
15
DLRM
An implementation of a deep learning recommendation model (DLRM)
...It includes data loaders for standard benchmarks (like Criteo), training scripts, evaluation tools, and capabilities like mixed precision, gradient compression, and memory fusion to maximize throughput. -
16
xFormers
Hackable and optimized Transformers building blocks
xformers is a modular, performance-oriented library of transformer building blocks, designed to allow researchers and engineers to compose, experiment, and optimize transformer architectures more flexibly than monolithic frameworks. It abstracts components like attention layers, feedforward modules, normalization, and positional encoding, so you can mix and match or swap optimized kernels easily. One of its key goals is efficient attention: it supports dense, sparse, low-rank, and... -
17
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
...Users write Python-like code (via DataFrame APIs or SQL strings) to express their transformations; behind the scenes, tasks are scheduled (often via Ray) and pushed into DuckDB instances operating on partitioned data. Because the storage layer (3FS) is optimized for random access and high throughput, smallpond can shuffle data, repartition, and manage intermediate results across nodes. -
18
Ring
Ring is a reasoning MoE LLM provided and open-sourced by InclusionAI
Ring is a reasoning Mixture-of-Experts (MoE) large language model (LLM) developed by inclusionAI. It is built from or derived from Ling. Its design emphasizes reasoning, efficiency, and modular expert activation. In its “flash” variant (Ring-flash-2.0), it optimizes inference by activating only a subset of experts. It applies reinforcement learning/reasoning optimization techniques. Its architectures and training approaches are tuned to enable efficient and capable reasoning performance.... -
19
Linera Protocol
Main repository for the Linera protocol
...The protocol supports dynamic reconfiguration: shards can split, merge, or reassign workloads while preserving correctness. By decoupling domain logic from global consensus, Linera can support modular upgrades, domain-specific smart contracts, and high throughput across partitions. -
20
Fastify
Fast and low overhead web framework, for Node.js
...Fastify ships with thoughtful defaults like structured logging via Pino and AJV for schema validation, and it has first-class TypeScript types. The result is a server stack that scales to high throughput while remaining easy to reason about in production. -
21
Isaac ROS Visual SLAM
Visual SLAM/odometry package based on NVIDIA-accelerated cuVSLAM
Discover a faster, easier way to build advanced AI robotics applications with the NVIDIA Isaac™ ROS collection of accelerated computing packages and AI models, bringing NVIDIA acceleration to ROS developers everywhere. Isaac ROS Visual SLAM provides a high-performance, best-in-class ROS 2 package for VSLAM (visual simultaneous localization and mapping). This package uses one or more stereo cameras and optionally an IMU to estimate odometry as an input to navigation. It is GPU-accelerated to... -
22
ParallelStencil.jl
Package for writing high-level code for parallel stencil computations
...For example, a 2-D shallow ice solver presented at JuliaCon 2020 [1] achieved a nearly 20 times better performance than a corresponding GPU Array programming implementation; in absolute terms, it reached 70% of the theoretical upper performance bound of the used Nvidia P100 GPU, as defined by the effective throughput metric, T_eff. ParallelStencil relies on the native kernel programming capabilities of CUDA.jl and AMDGPU.jl and on Base.Threads for high-performance computations on GPUs and CPUs, respectively. It is seamlessly interoperable with ImplicitGlobalGrid.jl, which renders the distributed parallelization of stencil-based GPU and CPU apps. -
23
Devtron
Tool integration platform for Kubernetes
...Provides deployment metrics like; deployment frequency, lead time, change failure rate, and mean-time recovery. Seamlessly integrates with Grafana for continuous application metrics like CPU and memory usage, status code, throughput, and latency on the dashboard. -
24
DALI
A GPU-accelerated library containing highly optimized building blocks
...DALI addresses the problem of the CPU bottleneck by offloading data preprocessing to the GPU. Additionally, DALI relies on its own execution engine, built to maximize the throughput of the input pipeline. -
25
ElasticJob
Distributed scheduled job framework
...It uses a unified job API for each project. Developers only need code one time and can deploy at will. Support job sharding and high availability in distributed system. Scale out for throughput and efficiency improvement. Job processing capacity is flexible and scalable with the allocation of resources. Execute job on suitable time and assigned resources. Aggregation same job to same job executor. Append resources to newly assigned jobs dynamically. Using ElasticJob can make developers no longer worry about the non-functional requirements such as jobs scale out, so that they can focus more on business coding.
