Search Results for "throughput" - Page 4
Sort By:
Showing 389 open source projects for "throughput"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild, govern, and optimize agents and models with Gemini Enterprise Agent Platform.
-
Stop Storing Third-Party Tokens in Your DatabaseRolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
-
1
EnvPool
C++-based high-performance parallel environment execution engine
EnvPool is a fast, asynchronous, and parallel RL environment library designed for scaling reinforcement learning experiments. Developed by SAIL at Singapore, it leverages C++ backend and Python frontend for extremely high-speed environment interaction, supporting thousands of environments running in parallel on a single machine. It's compatible with Gymnasium API and RLlib, making it suitable for scalable training pipelines. -
2
Bowtie 2
A fast and sensitive gapped read aligner
...It is widely used in bioinformatics pipelines for RNA-seq, DNA-seq, metagenomics, variant analysis, and other sequencing-based research tasks. Overall, Bowtie 2 remains a foundational command-line tool for high-throughput sequence alignment and reproducible computational biology workflows. -
3
Postgres Operator
Postgres operator creates and manages PostgreSQL clusters
The Postgres Operator delivers easy-to-run highly available PostgreSQL clusters on Kubernetes (K8s) powered by Patroni. It is configured only through Postgres manifests (CRDs) to ease integration into automated CI/CD pipelines with no access to Kubernetes API directly, promoting infrastructure as code vs manual operations. -
4
TensorRT LLM
TensorRT LLM provides users with an easy-to-use Python API
...It provides a Python-based API built on top of PyTorch that allows developers to define, customize, and deploy LLMs efficiently across a variety of hardware configurations, from single GPUs to large multi-node clusters. The library focuses on maximizing throughput and minimizing latency through advanced techniques such as quantization, custom attention kernels, and optimized memory management strategies. It includes support for cutting-edge inference methods like speculative decoding and inflight batching, enabling real-time and large-scale AI applications. TensorRT-LLM integrates seamlessly with NVIDIA’s broader inference ecosystem, including Triton Inference Server and distributed deployment frameworks, making it suitable for production environments. -
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
5
Pocket TTS
A TTS that fits in your CPU (and pocket)
...Because it is CPU-oriented, it fits well in server environments where GPU access is limited, in desktop apps, or in edge deployments where simplicity matters more than maximum throughput. It also emphasizes developer ergonomics, providing a straightforward API surface that can be integrated into pipelines, assistants, accessibility tools, or batch generation scripts. -
6
Supertonic
Lightning-fast, on-device TTS, running natively via ONNX
...The core model is highly compact at around 66 million parameters, yet benchmarks show it can generate speech up to 167× faster than real time on modern consumer hardware and significantly outpace popular cloud TTS APIs in throughput and real-time factor. Supertonic is designed to handle real-world text gracefully, including numbers, dates, currency symbols, abbreviations, and technical units, without requiring heavy pre-processing or custom text normalization. The repository provides complete reference implementations across many programming ecosystems—Python, Node.js, browser (WebGPU/WASM), Java, C++, C#, Go, Swift, iOS, Rust, and Flutter. -
7
Solon
Java enterprise application development framework
Solon is a full-scenario Java enterprise application framework that positions itself as a lean, high-performance alternative to heavy stacks. It advertises large concurrency gains, lower memory use, much faster startup, and dramatically smaller packages while remaining compatible from Java 8 through Java 24. The framework focuses on restrained APIs and an open ecosystem, with modules that cover web, data, cloud, and microservice patterns. Its messaging emphasizes “replaceable Spring”... -
8
GenStage
Producer and consumer actors with back-pressure for Elixir
...Because stages are OTP processes, you gain fault tolerance, supervised restarts, and concurrency tuned via configurable demand and partitioning. GenStage underpins higher-level libraries like Flow and Broadway, but it can also be used directly for custom pipelines where timing and throughput matter. Its clear separation of concerns encourages testable, composable stages that can be rearranged as requirements evolve. In production, this leads to predictable, resilient dataflows for event ingestion, batching, and parallel processing. -
9
Dragonfly
A modern replacement for Redis and Memcached
...Designed to fully utilize the power of modern cloud hardware and deliver on the data demands of modern applications, Dragonfly frees developers from the limits of traditional in-memory data stores. The power of modern cloud hardware can never be realized with legacy software. Dragonfly is optimized for modern cloud computing, delivering 25x more throughput and 12x lower snapshotting latency when compared to legacy in-memory data stores like Redis, making it easy to deliver the real-time experience your customers expect. Scaling Redis workloads is expensive due to their inefficient, single-threaded model. Dragonfly is far more compute and memory efficient, resulting in up to 80% lower infrastructure costs. ... -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
10
frugally-deep
A lightweight header-only library for using Keras (TensorFlow) models
...Utterly ignores even the most powerful GPU in your system and uses only one CPU core per prediction. Quite fast on one CPU core, and you can run multiple predictions in parallel, thus utilizing as many CPUs as you like to improve the overall prediction throughput of your application/pipeline. -
11
Stable Diffusion WebUI Forge
Stable Diffusion WebUI Forge is a platform on top of Stable Diffusion
Stable Diffusion WebUI Forge is a performance- and feature-oriented fork of the popular AUTOMATIC1111 interface that experiments with new backends, memory optimizations, and UX improvements. It targets heavy users and researchers who push large models, control nets, and high-resolution pipelines where default settings can become bottlenecks. The fork typically introduces toggles for scheduler behavior, attention implementations, caching, and precision modes to reach better speed or quality... -
12
workerpool
Offload tasks to a pool of workers on node.js and in the browser
...It supports task queuing, dynamic worker scaling, timeouts, transferable objects, and proxy-based APIs for interacting with worker functions as if they were local calls. workerpool is designed to improve application responsiveness and throughput in workloads involving heavy computation, data processing, or asynchronous execution. The library works across multiple environments, supporting Web Workers in browsers and Worker Threads or child processes in Node.js. Its architecture emphasizes ease of integration, scalability, and efficient concurrency management for modern JavaScript applications. -
13
VidGear
A High-performance cross-platform Video Processing Python framework
...The framework is built around modular components called “gears,” each responsible for tasks such as video capture, streaming, encoding, and network transmission. It supports multi-threaded and asynchronous operations, enabling low-latency processing and efficient handling of high-throughput video streams. VidGear is designed to handle a wide range of use cases, including live streaming, video stabilization, screencasting, and distributed video systems. Its emphasis on simplicity allows developers to implement advanced multimedia pipelines with minimal code while maintaining performance and flexibility. -
14
NeMo Retriever Library
Document content and metadata extraction microservice
...The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient handling of large datasets. It supports multiple extraction strategies for different document formats, balancing accuracy and throughput depending on the use case. Additionally, it can generate embeddings for extracted content and integrate with vector databases like Milvus, making it well-suited for retrieval-augmented generation pipelines. -
15
UCall
Up to 100x Faster FastAPI. JSON-RPC with io_uring, SIMDJSON
Most modern networking is built either on slow and ambiguous REST APIs or unnecessarily complex gRPC. FastAPI, for example, looks very approachable. We aim to be equally or even simpler to use. It takes over a millisecond to handle a trivial FastAPI call on a recent 8-core CPU. In that time, light could have traveled 300 km through optics to the neighboring city or country, in my case. How does UCall compare to FastAPI and gRPC? How can a tiny pet-project with just a couple thousand lines of... -
16
Scweet
Scrape tweets, profiles, followers and following from Twitter/X
Scweet is a Python-based Twitter/X scraping library and CLI designed to collect tweets, profile timelines, followers, following lists, and user profile data without requiring the official Twitter/X API or a developer account. Instead of depending on deprecated unauthenticated scraping methods, it works by using X’s web GraphQL API together with authenticated browser cookies, which gives it a more current and practical approach for data extraction. The project supports a broad set of... -
17
uzu
A high-performance inference engine for AI models
...The engine implements a hybrid architecture in which model layers can be executed either as custom GPU kernels or through Apple’s MPSGraph API, allowing it to balance performance and compatibility depending on the workload. By utilizing Apple’s unified memory architecture, uzu reduces memory copying overhead and improves inference throughput for local AI workloads. The system includes a simple high-level API that enables developers to run models, create inference sessions, and generate outputs with minimal configuration. -
18
MaxText
A simple, performant and scalable Jax LLM
...It is optimized to run efficiently on Google Cloud TPUs and GPUs, enabling researchers and engineers to train models ranging from small experiments to extremely large distributed workloads. The framework focuses on simplicity while still supporting advanced techniques such as model sharding, distributed computation, and high-throughput training pipelines. MaxText includes ready-to-use configurations and reproducible training examples that help developers understand how to deploy large-scale AI workloads with modern machine learning infrastructure. -
19
bitnet.cpp
Official inference framework for 1-bit LLMs
...BitNet is built to scale across architectures, with configurable kernels and tiling strategies that adapt to different hardware, and it supports large models with impressive throughput even on modest resources. -
20
GLM-OCR
Accurate × Fast × Comprehensive
GLM-OCR is an open-source multimodal optical character recognition (OCR) model built on a GLM-V encoder–decoder foundation that brings robust, accurate document understanding to complex real-world layouts and modalities. Designed to handle text recognition, table parsing, formula extraction, and general information retrieval from documents containing mixed content, GLM-OCR excels across major benchmarks while remaining highly efficient with a relatively compact parameter size (~0.9B),... -
21
MsQuic
Cross-platform, C implementation of the IETF QUIC protocol
...QUIC has many benefits when compared to existing "TLS over TCP" scenarios. MsQuic has several features that differentiate it from other QUIC implementations. Optimized for client and server. Optimized for maximal throughput and minimal latency. Asynchronous IO. Receive side scaling (RSS) support. UDP send and receive coalescing support. All packets are encrypted and handshake is authenticated with TLS 1.3. Parallel streams of (reliable and unreliable) application data. Exchange application data in the first round trip (0-RTT). Improved congestion control and loss recovery. ... -
22
listmonk
High performance, self-hosted, newsletter and mailing list manager
...Use template expressions, logic, and 100+ functions in subject lines and content. Write HTML e-mails in a WYSIWYG editor, Markdown, raw syntax-highlighted HTML, or just plain text. Highly configurable, multi-threaded, high-throughput multi-SMTP e-mail queues for super fast campaign delivery. -
23
Orpheus TTS
Towards Human-Sounding Speech
...The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research preview, and includes data-processing scripts so users can train or finetune their own variants. Inference is provided through a Python package that uses vLLM under the hood for high-throughput, low-latency generation, including streaming examples that show how to generate audio chunks in real time. The maintainers provide Colab notebooks, a standardized prompting format, and one-click deployment via Baseten for production-grade, FP8/FP16 optimized inference with ~200 ms streaming latency. -
24
Edgee
AI gateway with token compression for Claude Code, Codex, and more
Edgee is an edge-native execution platform designed to run AI-driven logic and data processing directly at the network edge, reducing latency and improving responsiveness for modern applications. It enables developers to deploy functions and workflows closer to users, allowing real-time processing without relying heavily on centralized cloud infrastructure. The platform is built to support event-driven architectures, where actions are triggered by incoming requests, user behavior, or... -
25
EFAK-AI
A AI-Driven, Distributed and high-performance monitoring system
EFAK (Eagle For Apache Kafka) is an open-source monitoring and management platform designed to provide comprehensive visibility and operational control over Apache Kafka clusters through a unified web interface. The project focuses on simplifying Kafka administration by offering real-time insights into cluster health, performance metrics, and consumer activity, allowing engineers to quickly diagnose issues and optimize system behavior. It integrates advanced features such as intelligent...
