Search Results for "latency"
Sort By:
Showing 125 open source projects for "latency"
View related business solutions-
8 Monitoring Tools in One APM. Install in 5 Minutes.AppSignal works out of the box for Ruby, Elixir, Node.js, Python, and more. 30-day free trial, no credit card required.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Sunshine
Self-hosted game stream host for Moonlight
Sunshine is an open-source self‑hosted cloud gaming server that implements NVIDIA’s GameStream protocol. Compatible with Moonlight clients across platforms, it supports low‑latency streaming via software or hardware encoding (AMD/Intel/NVIDIA) and offers a browser‑based control UI for pairing. -
2
DeepSpeed MII
MII makes low-latency and high-throughput inference possible
...While open-sourcing has democratized access to AI capabilities, their application is still restricted by two critical factors: inference latency and cost. DeepSpeed-MII is a new open-source python library from DeepSpeed, aimed towards making low-latency, low-cost inference of powerful models not only feasible but also easily accessible. MII offers access to the highly optimized implementation of thousands of widely used DL models. MII-supported models achieve significantly lower latency and cost compared to their original implementation. -
3
RealtimeSTT
A robust, efficient, low-latency speech-to-text library
RealtimeSTT is a Python-based realtime speech-to-text engine emphasizing low latency, wake-word detection, voice activity detection, and automatic speech segmentation. It provides asynchronous callbacks, nanosecond-precision timestamps, and CLI tools, suitable for building voice assistants, meeting transcribers, or live caption systems. -
4
Selkies-GStreamer
Open-Source Low-Latency Accelerated Linux WebRTC HTML5 Remote Desktop
selkies-gstreamer is a GStreamer-based media streaming component used in the Selkies project, a cloud-native platform designed for interactive desktop and application streaming. This module acts as a high-performance media pipeline that captures video, encodes it with low latency, and streams it via WebRTC to client browsers. It is optimized for GPU-accelerated encoding and integrates with Kubernetes-based deployments to enable scalable, real-time remote desktop sessions. This component plays a critical role in delivering smooth, responsive experiences for cloud-based workstations, gaming, or visualization tools. -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
Orpheus TTS
Towards Human-Sounding Speech
...The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research preview, and includes data-processing scripts so users can train or finetune their own variants. Inference is provided through a Python package that uses vLLM under the hood for high-throughput, low-latency generation, including streaming examples that show how to generate audio chunks in real time. The maintainers provide Colab notebooks, a standardized prompting format, and one-click deployment via Baseten for production-grade, FP8/FP16 optimized inference with ~200 ms streaming latency. -
6
FlashInfer
FlashInfer: Kernel Library for LLM Serving
FlashInfer is a kernel library designed to enhance the serving of Large Language Models (LLMs) by optimizing inference performance. It provides a high-performance framework that integrates seamlessly with existing systems, aiming to reduce latency and improve efficiency in LLM deployments. FlashInfer supports various hardware architectures and is built to scale with the demands of production environments. -
7
optillm
Optimizing inference proxy for LLMs
...It serves as an OpenAI API-compatible proxy, allowing for seamless integration into existing workflows while optimizing inference processes. OptiLLM aims to reduce latency and resource consumption during LLM inference. -
8
hls4ml
Machine learning on FPGAs using HLS
hls4ml is an open-source framework that enables machine learning models to be implemented directly on hardware such as FPGAs and ASICs using high-level synthesis techniques. The system converts trained neural network models from common machine learning frameworks into hardware description code suitable for ultra-low-latency inference. This approach allows machine learning algorithms to run directly on specialized hardware, making them suitable for applications that require extremely fast response times and minimal power consumption. The framework was originally developed for high-energy physics experiments where real-time decision systems must process large volumes of data with strict latency constraints. ... -
9
Qbot
AI-powered Quantitative Investment Research Platform
...It bundles a lightweight GUI client (built with wxPython) and a modular backend so researchers can iterate on strategies, run batch backtests, and validate ideas in a near-real simulated environment that models latency and slippage. The project places special emphasis on AI-driven strategies — including supervised learning, reinforcement learning and multi-factor models — and offers a “model zoo” and example strategies to help users get started. For evaluation and analysis, Qbot integrates reporting and visualization (tearsheets, metrics) so you can compare performance across runs and inspect trade-level behavior. ... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
OpenJarvis
Personal AI, On Personal Devices
...The framework provides shared primitives for building local-first agents, along with evaluation tools that measure performance using metrics such as energy consumption, latency, cost, and accuracy. OpenJarvis integrates with local inference engines like Ollama, vLLM, SGLang, and llama.cpp to run language models directly on personal hardware. It also includes a learning loop that allows models to improve over time using locally generated interaction traces. By prioritizing local execution and efficiency, OpenJarvis aims to provide a foundation for privacy-preserving personal AI assistants. -
11
Open Vision Agents by Stream
Build Vision Agents quickly with any model or video provider
...It focuses on combining video understanding models, such as YOLO and Roboflow based detectors, with real time large language models like OpenAI Realtime and Gemini Live to create interactive experiences. The framework uses Stream’s ultra low latency edge network so agents can join sessions quickly and maintain very low audio and video latency while processing frames and generating responses. Developers work with an agent abstraction that connects video edge providers, LLMs, and processors into pipelines, making it easier to orchestrate tasks like object detection, pose estimation, and conversational guidance. ... -
12
MemOS
AI memory OS for LLM and Agent systems
...By abandoning some of the historical assumptions of Unix-style operating systems, MemOS attempts to unlock new performance and scalability tradeoffs for applications that need high throughput and low latency on memory-intensive workloads. -
13
8mb.local
Ffree local self hosted video compressor webui
8mb.local is a lightweight, self-hosted local network status dashboard and utilities hub designed to help users monitor devices and services connected to their home or small business network. The platform presents a simple yet informative web dashboard that dynamically updates with device statuses, IP addresses, and latency information, giving users a centralized view of network health without complex dashboards. It also includes basic tools such as ping monitoring, service reachability checks, and port scanning, enabling users to diagnose connection issues quickly and without external services. The emphasis is on minimalism and performance, so it runs efficiently on low-power devices like Raspberry Pis or small containers, making it ideal for home labs or edge deployments. 8mb.local can also notify users of offline devices or connection drops through configurable alerts, helping preempt outages before they become serious. -
14
LingBot-World
Advancing Open-source World Models
...LingBot-World supports long-term temporal consistency, maintaining coherent scenes and interactions over minute-level horizons. With real-time interactivity and sub-second latency at 16 FPS, it is well-suited for interactive applications and rapid experimentation. The project is fully open-access, releasing both code and models to help bridge the gap between closed and open world-model systems. LingBot-World empowers researchers and developers in areas such as content creation, gaming, robotics, and embodied AI learning. -
15
DeepSpeed
Deep learning optimization library: makes distributed training easy
...Achieve excellent system throughput and efficiently scale to thousands of GPUs 3. Train/Inference on resource constrained GPU systems 4. Achieve unprecedented low latency and high throughput for inference 5. Achieve extreme compression for an unparalleled inference latency and model size reduction with low costs DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. ... -
16
RF-DETR
RF-DETR is a real-time object detection and segmentation
...The model is designed to detect objects and segment them within images or video streams using a unified detection pipeline. RF-DETR emphasizes strong performance across both accuracy and latency benchmarks, allowing developers to deploy high-quality detection models in applications that require immediate processing such as robotics, autonomous systems, and industrial inspection. The repository includes Python packages, training scripts, and model configurations that enable researchers and engineers to train and deploy detection models on custom datasets. -
17
Ultravox
Fast multimodal LLM for real-time voice interaction and AI apps
...Internally, it leverages pretrained language models and speech encoders, with a multimodal adapter that integrates both modalities for inference and training. Ultravox is optimized for low latency, achieving fast response times suitable for interactive voice agents and real-time applications. It supports use cases such as conversational AI agents, speech-to-speech translation, and analysis of spoken audio content. Ultravox also includes tooling and configuration systems for training, evaluation, and dataset integration. -
18
Fun Audio Chat
Large Audio Language Model built for natural interactions
...The system supports dynamic audio input and output, meaning it can handle different voices, tones, and conversational contexts without forcing users into typed interactions. With real-time streaming, it minimizes latency and delivers responses quickly, making it suitable for applications where responsiveness matters, such as interactive demos, accessibility tools, and conversational games. -
19
Memobase
Fast backend for long-term AI user memory via structured profiles
...The system focuses on three principal performance metrics: high search performance, reduced large language model (LLM) costs through batch processing techniques, and low latency with minimal SQL operations. Memobase supports integration with existing LLM workflows via APIs and SDKs (including Python, Node, and Go), making it easy to adopt within diverse application stacks. -
20
ElevenLabs Python
The official Python SDK for the ElevenLabs API
...The library wraps the HTTP API into a typed Python client, so you can perform text-to-speech, streaming, voice cloning, voice management, and agents-related operations with simple method calls. It exposes ElevenLabs’ main models such as Eleven Multilingual v2, Eleven Flash v2.5, and Eleven Turbo v2.5, each targeting different trade-offs between latency, cost, and quality. The SDK is designed for quick setup: after installing the package and setting an API key, you can generate speech in multiple languages and play or process the resulting audio bytes. It includes helper utilities (like play and stream) so you can either play audio locally or integrate it into your own playback or networking pipeline. -
21
RealtimeTTS
Converts text to speech in realtime
RealtimeTTS is a low-latency text-to-speech library built for real-time applications such as voice chat with LLMs, assistants, and interactive tools. It is designed around a streaming model: you can feed it text incrementally (for example, as an LLM responds) and get audio output almost immediately, which keeps end-to-end latency very low. The library is engine-agnostic and plugs into a wide range of cloud and local TTS systems, including OpenAI, ElevenLabs, Azure, Coqui, Piper, StyleTTS2, Edge TTS, Google TTS, system TTS and others, so you can swap providers without rewriting your pipeline. ... -
22
Frigate
NVR with realtime local object detection for IP cameras
Frigate - NVR With Realtime Object Detection for IP Cameras A complete and local NVR designed for Home Assistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras. Use of a Google Coral Accelerator is optional, but highly recommended. The Coral will outperform even the best CPUs and can process 100+ FPS with very little overhead. -
23
Parallax
Parallax is a distributed model serving framework
...A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
24
FireRedTTS-2
Long-form streaming TTS system for multi-speaker dialogue generation
FireRedTTS2 is a next-generation open-source text-to-speech (TTS) system focused on long-form, streaming speech synthesis for multi-speaker dialogue, delivering stable natural speech with context-aware prosody and reliable speaker transitions that support real-time and conversational applications. It features a specialized streaming speech tokenizer and a dual-transformer architecture that enables low latency and high-quality synthesis, making it suitable for interactive systems like chatbots, podcasts, and applications where dynamic turn-taking between speakers is essential. FireRedTTS2 supports multilingual output and speaker flexibility, enabling scenarios that involve language switching, cross-lingual voice cloning, and expressive dialogue generation that maintains consistency over longer utterances. -
25
MobileCLIP
Implementation of "MobileCLIP" CVPR 2024
...It includes an iOS demo app and Core ML artifacts to showcase practical, offline photo search and classification on iPhone-class hardware. Project notes highlight latency/accuracy trade-offs, with MobileCLIP2 variants matching or surpassing larger baselines at notably lower parameter counts and runtime on mobile devices. A companion “mobileclip-dr” repository details large-scale, distributed data-generation pipelines used to reinforce datasets across billions of samples on thousands of GPUs. Overall, MobileCLIP emphasizes end-to-end practicality: scalable training, deployable models, and consumer-grade demos.
