Search Results for "throughput" - Page 4
Sort By:
Showing 350 open source projects for "throughput"
View related business solutions-
Gemini 3 and 200+ AI Models on One PlatformBuild, govern, and optimize agents and models with Gemini Enterprise Agent Platform.
-
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
1
TensorRT LLM
TensorRT LLM provides users with an easy-to-use Python API
...It provides a Python-based API built on top of PyTorch that allows developers to define, customize, and deploy LLMs efficiently across a variety of hardware configurations, from single GPUs to large multi-node clusters. The library focuses on maximizing throughput and minimizing latency through advanced techniques such as quantization, custom attention kernels, and optimized memory management strategies. It includes support for cutting-edge inference methods like speculative decoding and inflight batching, enabling real-time and large-scale AI applications. TensorRT-LLM integrates seamlessly with NVIDIA’s broader inference ecosystem, including Triton Inference Server and distributed deployment frameworks, making it suitable for production environments. -
2
Solon
Java enterprise application development framework
Solon is a full-scenario Java enterprise application framework that positions itself as a lean, high-performance alternative to heavy stacks. It advertises large concurrency gains, lower memory use, much faster startup, and dramatically smaller packages while remaining compatible from Java 8 through Java 24. The framework focuses on restrained APIs and an open ecosystem, with modules that cover web, data, cloud, and microservice patterns. Its messaging emphasizes “replaceable Spring”... -
3
GenStage
Producer and consumer actors with back-pressure for Elixir
...Because stages are OTP processes, you gain fault tolerance, supervised restarts, and concurrency tuned via configurable demand and partitioning. GenStage underpins higher-level libraries like Flow and Broadway, but it can also be used directly for custom pipelines where timing and throughput matter. Its clear separation of concerns encourages testable, composable stages that can be rearranged as requirements evolve. In production, this leads to predictable, resilient dataflows for event ingestion, batching, and parallel processing. -
4
Stable Diffusion WebUI Forge
Stable Diffusion WebUI Forge is a platform on top of Stable Diffusion
Stable Diffusion WebUI Forge is a performance- and feature-oriented fork of the popular AUTOMATIC1111 interface that experiments with new backends, memory optimizations, and UX improvements. It targets heavy users and researchers who push large models, control nets, and high-resolution pipelines where default settings can become bottlenecks. The fork typically introduces toggles for scheduler behavior, attention implementations, caching, and precision modes to reach better speed or quality... -
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
5
workerpool
Offload tasks to a pool of workers on node.js and in the browser
...It supports task queuing, dynamic worker scaling, timeouts, transferable objects, and proxy-based APIs for interacting with worker functions as if they were local calls. workerpool is designed to improve application responsiveness and throughput in workloads involving heavy computation, data processing, or asynchronous execution. The library works across multiple environments, supporting Web Workers in browsers and Worker Threads or child processes in Node.js. Its architecture emphasizes ease of integration, scalability, and efficient concurrency management for modern JavaScript applications. -
6
NeMo Retriever Library
Document content and metadata extraction microservice
...The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient handling of large datasets. It supports multiple extraction strategies for different document formats, balancing accuracy and throughput depending on the use case. Additionally, it can generate embeddings for extracted content and integrate with vector databases like Milvus, making it well-suited for retrieval-augmented generation pipelines. -
7
UCall
Up to 100x Faster FastAPI. JSON-RPC with io_uring, SIMDJSON
Most modern networking is built either on slow and ambiguous REST APIs or unnecessarily complex gRPC. FastAPI, for example, looks very approachable. We aim to be equally or even simpler to use. It takes over a millisecond to handle a trivial FastAPI call on a recent 8-core CPU. In that time, light could have traveled 300 km through optics to the neighboring city or country, in my case. How does UCall compare to FastAPI and gRPC? How can a tiny pet-project with just a couple thousand lines of... -
8
Scweet
Scrape tweets, profiles, followers and following from Twitter/X
Scweet is a Python-based Twitter/X scraping library and CLI designed to collect tweets, profile timelines, followers, following lists, and user profile data without requiring the official Twitter/X API or a developer account. Instead of depending on deprecated unauthenticated scraping methods, it works by using X’s web GraphQL API together with authenticated browser cookies, which gives it a more current and practical approach for data extraction. The project supports a broad set of... -
9
uzu
A high-performance inference engine for AI models
...The engine implements a hybrid architecture in which model layers can be executed either as custom GPU kernels or through Apple’s MPSGraph API, allowing it to balance performance and compatibility depending on the workload. By utilizing Apple’s unified memory architecture, uzu reduces memory copying overhead and improves inference throughput for local AI workloads. The system includes a simple high-level API that enables developers to run models, create inference sessions, and generate outputs with minimal configuration. -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
MaxText
A simple, performant and scalable Jax LLM
...It is optimized to run efficiently on Google Cloud TPUs and GPUs, enabling researchers and engineers to train models ranging from small experiments to extremely large distributed workloads. The framework focuses on simplicity while still supporting advanced techniques such as model sharding, distributed computation, and high-throughput training pipelines. MaxText includes ready-to-use configurations and reproducible training examples that help developers understand how to deploy large-scale AI workloads with modern machine learning infrastructure. -
11
bitnet.cpp
Official inference framework for 1-bit LLMs
...BitNet is built to scale across architectures, with configurable kernels and tiling strategies that adapt to different hardware, and it supports large models with impressive throughput even on modest resources. -
12
GLM-OCR
Accurate × Fast × Comprehensive
GLM-OCR is an open-source multimodal optical character recognition (OCR) model built on a GLM-V encoder–decoder foundation that brings robust, accurate document understanding to complex real-world layouts and modalities. Designed to handle text recognition, table parsing, formula extraction, and general information retrieval from documents containing mixed content, GLM-OCR excels across major benchmarks while remaining highly efficient with a relatively compact parameter size (~0.9B),... -
13
listmonk
High performance, self-hosted, newsletter and mailing list manager
...Use template expressions, logic, and 100+ functions in subject lines and content. Write HTML e-mails in a WYSIWYG editor, Markdown, raw syntax-highlighted HTML, or just plain text. Highly configurable, multi-threaded, high-throughput multi-SMTP e-mail queues for super fast campaign delivery. -
14
Orpheus TTS
Towards Human-Sounding Speech
...The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research preview, and includes data-processing scripts so users can train or finetune their own variants. Inference is provided through a Python package that uses vLLM under the hood for high-throughput, low-latency generation, including streaming examples that show how to generate audio chunks in real time. The maintainers provide Colab notebooks, a standardized prompting format, and one-click deployment via Baseten for production-grade, FP8/FP16 optimized inference with ~200 ms streaming latency. -
15
Edgee
AI gateway with token compression for Claude Code, Codex, and more
Edgee is an edge-native execution platform designed to run AI-driven logic and data processing directly at the network edge, reducing latency and improving responsiveness for modern applications. It enables developers to deploy functions and workflows closer to users, allowing real-time processing without relying heavily on centralized cloud infrastructure. The platform is built to support event-driven architectures, where actions are triggered by incoming requests, user behavior, or... -
16
EFAK-AI
A AI-Driven, Distributed and high-performance monitoring system
EFAK (Eagle For Apache Kafka) is an open-source monitoring and management platform designed to provide comprehensive visibility and operational control over Apache Kafka clusters through a unified web interface. The project focuses on simplifying Kafka administration by offering real-time insights into cluster health, performance metrics, and consumer activity, allowing engineers to quickly diagnose issues and optimize system behavior. It integrates advanced features such as intelligent... -
17
OpenAI Forward
An efficient forwarding service designed for LLMs
...The project can proxy both local and cloud-hosted language model services, which makes it useful for teams that want a single control layer regardless of whether they are using something like LocalAI or a hosted provider compatible with OpenAI-style APIs. A major emphasis of the repository is asynchronous performance, using tools such as uvicorn, aiohttp, and asyncio to support high-throughput forwarding workloads. -
18
MyPerf4J
High performance Java APM. Powered by ASM
MyPerf4J is an open source Java performance monitoring and profiling tool designed to help developers and operations teams analyze application behavior with minimal overhead. It focuses on providing lightweight yet high-precision metrics collection for method execution times, throughput, and latency, making it particularly useful in high-performance production environments. The project emphasizes low performance impact by using efficient time recording and aggregation techniques, allowing continuous monitoring without significantly affecting application speed. MyPerf4J supports both real-time monitoring and offline analysis through log aggregation and visualization tools. ... -
19
WGCLOUD
Visibility into servers, applications, and infrastructure
...Built on a Spring Boot microservices foundation with an agent-server model, the system emphasizes rapid deployment, minimal configuration overhead, and automated operation for large-scale environments. It collects extensive host metrics such as CPU usage, temperature, memory utilization, disk performance, network throughput, and hardware health while also supporting monitoring of processes, containers, ports, and databases. The platform includes advanced operational capabilities such as web-based SSH access, batch command execution, and visual topology generation, making it more than a passive monitoring tool. wgcloud is intended for private, self-hosted deployments where organizations want full control over monitoring data without relying on external cloud services. -
20
Zvec
A lightweight, lightning-fast, in-process vector database
...Developed by Alibaba’s Tongyi Lab, it positions itself as the “SQLite of vector databases” by being easy to integrate, minimal in dependencies, and capable of handling high throughput with low latency on edge devices or small systems. Zvec excels at approximate nearest neighbor search and retrieval tasks that power features like semantic search, recommendation systems, and retrieval-augmented generation (RAG) setups. Its performance benchmarks show it achieving high queries-per-second and fast index build times compared to similar tools. ... -
21
SyncKit
Local-first collaboration SDK for React, Vue, and Svelte
...Synckit’s synchronization engine efficiently handles differential syncing — transferring only changed blocks rather than entire files — which reduces bandwidth usage and accelerates sync throughput. -
22
Loki Mode
Multi-agent autonomous startup system for Claude Code
...By supporting multiple AI providers (like Claude Code, OpenAI Codex CLI, and Google Gemini CLI), loki-mode dynamically selects and spawns only the needed agents for a given project, optimizing computational resources and task throughput. Its Reason-Act-Reflect-Verify (RARV) cycle with self-verification loops emphasizes quality and resilience, automating end-to-end development lifecycles. -
23
Marl
A hybrid thread / fiber task scheduler written in C++ 11
marl is a lightweight C++ job system that schedules tasks on a pool of worker threads and runs them using cooperative fibers for high throughput. By decoupling “tasks” from OS threads, it minimizes context switches and lets workloads with fine-grained synchronization scale efficiently. The API is intentionally small—post tasks, wait on fences, and express dependencies—so you can retrofit it into existing engines without adopting a heavyweight framework. Under the hood, fibers allow tasks to yield while waiting, keeping cores busy without blocking threads. ... -
24
RobotsTxt
The repository contains Google's robots.txt parser
This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled... -
25
Wireit
Wireit upgrades your npm/pnpm/yarn scripts to make them smarter
...It persists caches between runs and across CI so identical steps don’t redo expensive tasks like bundling or type-checking. The system also serializes or parallelizes scripts based on dependencies to avoid race conditions while maximizing throughput. Because it integrates at the script layer, it works with any underlying tool—TypeScript, Rollup, Jest, or bespoke commands—while remaining transparent to developers.
