Search Results for "throughput"
Sort By:
Showing 516 open source projects for "throughput"
View related business solutions-
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
1
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
2
Garnet
Garnet is a remote cache-store from Microsoft Research
Garnet is a remote cache‑store developed by Microsoft Research. It delivers high throughput and low‑latency performance, supports scalability via clustering (sharding, replication, key migration, checkpointing, failover, transactions), and seamlessly integrates with existing Redis clients. Garnet offers much better throughput and scalability with many client connections and small batches, relative to comparable open-source cache-stores, leading to cost savings for large apps and services. ... -
3
ntopng
Web-based Traffic and Security Network Traffic Monitoring
ntopng® is a web-based network traffic monitoring application released under GPLv3. It is the new incarnation of the original ntop written in 1998, and is now revamped in terms of performance, usability, and features. ntopng is a network traffic probe that provides 360° Network visibility, with its ability to gather traffic information from traffic mirrors, NetFlow exporters, SNMP devices, Firewall logs, and Intrusion Detection systems. ntopng has been written in a portable way in order to... -
4
LavinMQ
Ultra quick message queue and streaming server
...It implements AMQP 0-9-1 and MQTT 3.1.1, making it useful for applications that need reliable messaging, IoT communication, event processing, and backend service coordination. The server is written in Crystal, which helps it keep memory usage low while supporting high throughput. It is designed to handle very long queues, many client connections, and production workloads with minimal configuration. LavinMQ can be used as a lightweight alternative in RabbitMQ-compatible environments where AMQP-based clients are already common. Its main value is combining speed, low resource consumption, protocol compatibility, and operational simplicity in one open-source broker. -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
iperf3
iperf3: A TCP, UDP, and SCTP network bandwidth measurement tool
iperf3 is the official iperf3 repository, a network performance measurement tool for testing maximum achievable bandwidth on IP networks. It supports TCP, UDP, and SCTP testing, making it useful for a wide range of network troubleshooting and benchmarking scenarios. The tool reports throughput, bitrate, packet loss, and other measurement details for each test. iperf3 was redesigned from the original iperf with a smaller codebase, a cleaner architecture, and library functionality that can be used by other programs. It includes features such as zero-copy mode, optional JSON output, protocol tuning, buffer tuning, and timing controls. ... -
6
HikariCP
A solid, high-performance, JDBC connection pool at last
HikariCP is a high-performance JDBC connection pooling library designed to offer minimal overhead and maximum throughput. It aims for simplicity and stability, concentrating on fast connection acquisition and minimal latency under load. Its internals manage a pool of connections aggressively but smartly, with options such as leak detection, timeout thresholds, and adaptive connection retirement. Because it avoids heavy internal locking, it scales well in high-concurrency environments, making it a popular choice in microservices and high-throughput web backends. ... -
7
conflux-rust
The official Rust implementation of Conflux protocol
conflux-rust is the Rust implementation of Conflux, a high-performance Layer 1 blockchain designed to deliver high throughput without compromising decentralization or security. Conflux introduces a Tree-Graph consensus mechanism that allows parallel block processing while maintaining consensus integrity. This implementation focuses on performance and is compatible with the Ethereum Virtual Machine (EVM), enabling developers to deploy smart contracts written for Ethereum. -
8
ThingsBoard Message Queue (TBMQ)
Open-source, scalable, and fault-tolerant MQTT broker
TBMQ is a lightweight message broker built to support ThingsBoard's IoT platform, focusing on telemetry data streaming and device communication. It uses Kafka-compatible APIs and is optimized for high-throughput messaging, device scalability, and low-latency delivery. TBMQ is ideal for IoT backends needing MQTT or Kafka-style pub/sub infrastructure. -
9
ScaleLLM
A high-performance inference system for large language models
ScaleLLM is a high-performance inference system tailored for Large Language Models (LLMs), specifically designed for production environments. It focuses on optimizing inference processes to handle large-scale deployments efficiently, ensuring low latency and high throughput. ScaleLLM supports various LLM architectures and integrates with existing infrastructures, providing a scalable solution for deploying LLMs in real-world applications. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
DeepSpeed
Deep learning optimization library: makes distributed training easy
DeepSpeed is an easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference. With DeepSpeed you can: 1. Train/Inference dense or sparse models with billions or trillions of parameters 2. Achieve excellent system throughput and efficiently scale to thousands of GPUs 3. Train/Inference on resource constrained GPU systems 4. Achieve unprecedented low latency and high throughput for inference 5. Achieve extreme compression for an unparalleled inference latency and model size reduction with low costs DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. ... -
11
Parallax
Parallax is a distributed model serving framework
...A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
12
RTP-LLM
Alibaba's high-performance LLM inference engine for diverse apps
RTP-LLM is an open-source large language model inference acceleration engine developed by Alibaba to provide high-performance serving infrastructure for modern LLM deployments. The system focuses on improving throughput, latency, and resource utilization when running large models in production environments. It achieves this by implementing optimized GPU kernels, batching strategies, and memory management techniques tailored for transformer inference workloads. The framework is designed for large-scale AI services and is already used internally across several Alibaba platforms such as Taobao, Amap, and other business systems that rely on conversational or search-related AI services. ... -
13
LitServe
Minimal Python framework for scalable AI inference servers fast
...Unlike traditional serving tools that enforce rigid abstractions, LitServe focuses on flexibility by letting users control request handling, batching strategies, and output processing directly in Python. LitServe is built on top of FastAPI and extends it with AI-specific optimizations such as efficient multi-worker execution, which can significantly improve throughput. It includes built-in capabilities for batching, streaming responses, and automatic scaling across CPUs and GPUs, enabling high-performance deployments. -
14
Text Embeddings Inference
High-performance inference server for text embeddings models API layer
...It provides an API interface that allows developers to integrate embedding capabilities into applications without managing model internals directly. Text Embeddings Inference is optimized for throughput and low latency, enabling it to handle large volumes of requests reliably. It also emphasizes ease of deployment, often using containerization and configurable runtime options to adapt to different infrastructure setups. -
15
FlexLLMGen
Running large language models on a single GPU
...This design allows organizations to deploy powerful language models for high-volume tasks without the infrastructure costs typically associated with large-scale AI systems. The project is particularly useful for workloads that prioritize throughput over latency, including benchmarking experiments and large corpus analysis. -
16
MiMo-V2-Flash
MiMo-V2-Flash: Efficient Reasoning, Coding, and Agentic Foundation
...It uses an MoE setup where a very large total parameter count is available, but only a smaller subset is activated per token, which helps balance capability with runtime efficiency. The project positions the model for workflows that require tool use, multi-step planning, and higher throughput, rather than only single-turn chat. Architecturally, it highlights attention and prediction choices aimed at accelerating generation while preserving instruction-following quality in complex prompts. The repository typically serves as a launch point for running the model, understanding its intended use cases, and reproducing or extending its evaluation on reasoning and agent-style tasks. ... -
17
Olric
Distributed, in-memory key/value store and cache
A lightweight, distributed in-memory data store designed for key-value caching and ephemeral storage. -
18
MemOS
AI memory OS for LLM and Agent systems
...By abandoning some of the historical assumptions of Unix-style operating systems, MemOS attempts to unlock new performance and scalability tradeoffs for applications that need high throughput and low latency on memory-intensive workloads. -
19
Aeron
Efficient UDP unicast, UDP multicast, and IPC message transport
...Aeron Cluster provides support for fault-tolerant services as replicated state machines based on the Raft consensus algorithm. Performance is the key focus. A design goal for Aeron is to be the highest throughput with the lowest and most predictable latency of any messaging system. Aeron integrates with Simple Binary Encoding (SBE) for the best possible message encoding and decoding performance. Many of the data structures used in the creation of Aeron have been factored out to the Agrona project. -
20
Node RDKafka
Node.js bindings for librdkafka
A high-performance Node.js client for Apache Kafka, built on top of librdkafka, providing bindings for efficient Kafka message processing. -
21
go-web-framework-benchmark
Go web framework benchmark
...It can add configurable handler delay to simulate business work such as database calls, cache access, disk writes, socket activity, or microservice calls. The benchmark uses wrk and scripts to collect throughput, latency, allocation, concurrency, pipelining, and CPU-bound results. It is useful for developers who want repeatable framework comparisons that better reflect end-to-end web request behavior. -
22
Infinity
Low-latency REST API for serving text-embeddings
Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting all sentence-transformer models and frameworks. Infinity is developed under MIT License. Infinity powers inference behind Gradient.ai and other Embedding API providers. -
23
DeepSpeed MII
MII makes low-latency and high-throughput inference possible
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed. The Deep Learning (DL) open-source community has seen tremendous growth in the last few months. Incredibly powerful text generation models such as the Bloom 176B, or image generation model such as Stable Diffusion are now available to anyone with access to a handful or even a single GPU through platforms such as Hugging Face. -
24
Shardeum
Shardeum is an EVM based autoscaling blockchain
Shardeum is an EVM‑compatible layer‑1 blockchain platform that leverages dynamic state sharding to deliver linear scalability, consistently low transaction fees, strong decentralization, and high throughput for decentralized application developers. Shardeum is an innovative EVM-compliant blockchain platform that leverages dynamic state sharding to achieve unprecedented scalability. By implementing a sharding model, Shardeum ensures faster processing times and lower transaction costs without compromising security or decentralization. ... -
25
r2dbc-postgresql
Postgresql R2DBC Driver
r2dbc-postgresql is a reactive PostgreSQL driver for the JVM based on the R2DBC (Reactive Relational Database Connectivity) specification. It allows non-blocking, asynchronous access to PostgreSQL databases, making it ideal for reactive Java or Kotlin applications built with frameworks like Spring WebFlux.
