Search Results for "latency" - Page 5
Sort By:
Showing 661 open source projects for "latency"
View related business solutions-
Paessler - Monitor Your Whole Network in MinutesWaiting weeks for a monitoring rollout isn't an option when infrastructure doesn't stop running. PRTG's auto-discovery scans your network and suggests from over 200 pre-configured sensor types, so you're watching servers, applications and devices within minutes, not after a multi-week deployment. Enterprise-strength monitoring, without the enterprise complexity. Start your free trial today.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Parallax
Parallax is a distributed model serving framework
...A two-stage scheduling architecture determines how model layers are allocated to available hardware and how requests are routed across nodes during execution. This scheduling system optimizes latency, throughput, and hardware utilization even when nodes have different computational capabilities. The platform also supports model sharding and pipeline parallelism, allowing very large models to run across distributed resources. -
2
LangWatch
The platform for LLM evaluations and AI agent testing
LangWatch is an open-source observability and monitoring platform designed to help developers evaluate and improve applications built with large language models. The platform provides tools for tracking model interactions, analyzing prompt behavior, and identifying issues such as hallucinations, latency problems, or unexpected responses. By collecting telemetry data from AI applications, LangWatch allows developers to understand how their systems perform in real-world usage scenarios. The platform includes dashboards that visualize model behavior, enabling teams to monitor trends in response quality and reliability over time. It also provides evaluation tools that allow developers to test prompts and compare outputs across different models or configurations. ... -
3
Cloudflare Agents
Build and deploy AI Agents on Cloudflare
Cloudflare Agents is an open-source framework designed to help developers build, deploy, and manage AI agents that run at the network edge. It provides infrastructure for creating stateful, event-driven agents capable of real-time interaction while maintaining low latency through Cloudflare’s distributed platform. The project includes SDKs, templates, and deployment tooling that simplify the process of connecting agents to external APIs, storage systems, and workflows. Its architecture emphasizes persistent memory, enabling agents to maintain context across sessions and interactions. Developers can orchestrate complex behaviors using workflows and durable objects, making it suitable for production-grade autonomous systems. ... -
4
MemOS
AI memory OS for LLM and Agent systems
...By abandoning some of the historical assumptions of Unix-style operating systems, MemOS attempts to unlock new performance and scalability tradeoffs for applications that need high throughput and low latency on memory-intensive workloads. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
5
sshx
Fast, collaborative live terminal sharing over the web
...The interface supports an infinite canvas for arranging multiple terminal panes, real-time remote cursors, automatic reconnection, and predictive local echo to make typing feel fluid even with network latency. SSHX’s mesh networking architecture connects participants to the nearest server, facilitating fast interactions globally. Because it’s open source and cross-platform, it can be used on Linux, macOS, and Windows systems, and it integrates with CI/CD workflows to make collaborative debugging easier. -
6
Obsidian Visual Skills Pack
Generate Canvas, Excalidraw, and Mermaid diagrams from text
...Traditional approaches often dump entire files into a model’s context, which quickly exceeds token limits; LLM-TLDR instead indexes project structure, traces dependencies, and summarizes code in a way that preserves semantic relevance while shrinking input size by up to 95 %. This makes queries and analysis much faster and cheaper, with dramatic token savings and latency improvements for LLM-driven development workflows. The project supports multiple programming languages and includes utilities for warming an index and then generating LLM-ready contexts or summaries of specific parts of a project. -
7
GrowthBook
Open source feature flagging and AB testing platform
...It supports both self-hosted and cloud-hosted deployment models, giving organizations the flexibility to own their infrastructure or consume it as a managed service. The platform is designed for performance and scale: its SDKs are lightweight, supporting local evaluation to minimize latency, and it integrates deeply with existing data stacks so you can use your warehouse or analytics system as the source of truth. Experimentation in GrowthBook isn’t just toggles; its statistics engine supports advanced techniques like CUPED, Bayesian, and sequential testing, and control group checks so you can confidently measure impact. -
8
Fully Homomorphic Encryption
An FHE compiler for C++
...The workflow usually mirrors normal software development: write and test a cleartext implementation, run a simulator to validate logic and performance characteristics, then compile to an encrypted form to run with real FHE parameters. The project includes benchmarking harnesses, examples, and tutorials that demystify parameter selection, gate costs, and latency/throughput trade-offs. Multiple runtimes and backends are supported so teams can choose Boolean or integer schemes depending on accuracy and speed needs. -
9
MobileCLIP
Implementation of "MobileCLIP" CVPR 2024
...It includes an iOS demo app and Core ML artifacts to showcase practical, offline photo search and classification on iPhone-class hardware. Project notes highlight latency/accuracy trade-offs, with MobileCLIP2 variants matching or surpassing larger baselines at notably lower parameter counts and runtime on mobile devices. A companion “mobileclip-dr” repository details large-scale, distributed data-generation pipelines used to reinforce datasets across billions of samples on thousands of GPUs. Overall, MobileCLIP emphasizes end-to-end practicality: scalable training, deployable models, and consumer-grade demos. -
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
10
CoreNet
CoreNet: A library for training deep neural networks
CoreNet is Apple’s internal deep learning framework for distributed neural network training, designed for high scalability, low-latency communication, and strong hardware efficiency. It focuses on enabling large-scale model training across clusters of GPUs and accelerators by optimizing data flow and parallelism strategies. CoreNet provides abstractions for data, tensor, and pipeline parallelism, allowing models to scale without code duplication or heavy manual configuration. -
11
libSQL
libSQL is a fork of SQLite that is both Open Source
libSQL is an open-source, community-driven fork of SQLite maintained by Turso. It aims to modernize SQLite by optimizing for performance and availability, targeting distributed and low-latency applications. SQLite has solidified its place in modern technology stacks, embedded in nearly any computing device you can think of. Its open source nature and public domain availability make it a popular choice for modification to meet specific use cases. libSQL will always be able to ingest and write the SQLite file format. We would love to add extensions like encryption, and CRC that require the file to be changed. ... -
12
Simple Binary Encoding (SBE)
Simple Binary Encoding (SBE) - High Performance Message Codec
...The Java and C++ SBE implementations work very efficiently with the Aeron messaging system for low-latency and high-throughput communications. The Java SBE implementation has a dependency on Agrona for its buffer implementations. -
13
Frigate
NVR with realtime local object detection for IP cameras
Frigate - NVR With Realtime Object Detection for IP Cameras A complete and local NVR designed for Home Assistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras. Use of a Google Coral Accelerator is optional, but highly recommended. The Coral will outperform even the best CPUs and can process 100+ FPS with very little overhead. -
14
Apache RocketMQ
Distributed messaging and streaming platform with low latency
Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity and flexible scalability. Messaging patterns including publish/subscribe, request/reply and streaming. Financial grade transactional message. Built-in fault tolerance and high availability configuration options base on DLedger. A variety of cross language clients, such as Java, C/C++, Python, Go. -
15
TensorRT
C++ library for high performance inference on NVIDIA GPUs
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference. It includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for deep learning inference applications. TensorRT-based applications perform up to 40X faster than CPU-only platforms during inference. With TensorRT, you can optimize neural network models trained in all major frameworks, calibrate for lower precision with high accuracy, and deploy to hyperscale data centers, embedded, or automotive product platforms. ... -
16
Tarantool
Get your data in RAM, get compute close to data, enjoy the performance
...With Tarantool, you can replace the traditional bundle of database & cache and reduce operational costs. Tarantool is tolerant of write-heavy loads. It also allows keeping full-featured applications close to the data, thus reducing data access network latency to zero. The open-source Community Edition lets you develop applications and speed up a system in operation. It features synchronous replication, affords easy scalability, and includes tools to develop efficient applications. The Tarantool community helps with any practical questions regarding the Community Edition. Large enterprises have demanding requirements for information safety and security. ... -
17
ZLMediaKit
WebRTC/RTSP/RTMP/HTTP/HLS/HTTP-FLV/WebSocket
...It supports a wide range of protocols, including RTSP, RTMP, HLS, HTTP-FLV, WebSocket-FLV, GB28181, HTTP-TS, HTTP-fMP4, MP4, and WebRTC. The project is designed for protocol conversion, live streaming, low-latency playback, recording, forwarding, and large-scale client access. Its asynchronous, multithreaded network model makes it suitable for commercial streaming deployments with many concurrent connections. ZLMediaKit can be deployed directly as a complete media server or embedded through its standard C API as an SDK for other languages and systems. ... -
18
Jessibuca
Jessibuca is an open source pure H5 live streaming player
...It compiles native decoding libraries into JavaScript, enabling high-performance playback across desktop and mobile devices. The player supports modern streaming protocols and codecs, allowing it to handle real-time video feeds with low latency and stable rendering. Its architecture is modular, separating decoding, demuxing, and rendering to optimize performance and scalability. Jessibuca includes intelligent buffering and frame handling to prevent delay accumulation during long playback sessions. It also supports multiple playback instances simultaneously, making it suitable for monitoring dashboards or multi-stream environments. ... -
19
MiniCPM4
Ultra-Efficient LLMs on End Device
...Unlike traditional large-scale models that require extensive computational resources, MiniCPM4 focuses on delivering competitive reasoning and language capabilities while maintaining significantly lower latency and higher efficiency. It achieves this through optimized architectures, scalable training strategies, and techniques such as long-context pretraining and YaRN-based length extension, allowing it to handle sequences up to 128K tokens effectively. The model demonstrates strong performance across tasks such as long-text comprehension, reasoning, and general language generation, often outperforming similar-sized models in both speed and accuracy. ... -
20
RISC-V sandboxing library
The fastest RISC-V sandbox
RISC-V sandboxing library is a high-performance, ultra-low-latency RISC-V userspace emulator library written in modern C++, designed for embedding and sandboxing applications. Unlike full-system emulators, it focuses specifically on executing user-space programs, making it ideal for scenarios such as sandboxed execution, scripting engines, and high-performance server environments. The library is engineered to achieve extremely fast startup and execution times, with the ability to run large numbers of virtual machines concurrently with minimal resource usage. ... -
21
HelixDB
Graph-vector database for building unified AI backends fast
...It combines graph and vector data models, allowing developers to manage relationships and embeddings within the same system without relying on separate services. HelixDB is built from scratch in Rust and uses LMDB as its storage engine, enabling high performance and low-latency query execution. HelixDB also supports additional data formats such as key-value, document, and relational data, making it flexible for a wide range of backend architectures. A central feature of the project is its custom query language, HelixQL, which is fully type-safe and compiled to ensure reliability and correctness in production environments. ... -
22
DeepReasoning
High-performance API combining reasoning and creative AI models
...It emphasizes real-time streaming responses, where reasoning and final answers can be delivered efficiently in a single flow. Built with Rust, the backend focuses on performance, low latency, and efficient resource usage for demanding workloads. It follows a bring-your-own-key approach, giving users full control over their API credentials and data privacy. Additionally, it supports self-hosting and extensive configuration, making it suitable for developers who want flexibility and control. -
23
Korvus
Korvus is a search SDK that unifies the entire RAG pipeline
...By leveraging PostgresML and vector extensions such as pgvector, Korvus eliminates the need for external microservices typically used for AI search architectures, reducing both system complexity and latency. The architecture enables machine learning operations to occur directly in the database, minimizing data transfer between services and improving overall performance for large datasets. -
24
CAG
Cache-Augmented Generation: A Simple, Efficient Alternative to RAG
...This strategy allows the model to generate responses using the cached context directly, eliminating the need for repeated retrieval operations during runtime. As a result, the approach can significantly reduce latency and simplify system architecture compared with traditional RAG pipelines. The framework is particularly effective when the knowledge base is limited enough to fit within the extended context window of modern language models. -
25
LightLLM
LightLLM is a Python-based LLM (Large Language Model) inference
...Built primarily in Python, the project integrates optimization techniques and ideas from several leading open-source implementations, including FasterTransformer, vLLM, and FlashAttention, to accelerate token generation and reduce latency. LightLLM is designed to handle large-scale model workloads in production environments, supporting efficient batching and GPU utilization for fast inference across multiple requests. Its architecture allows models to be deployed with minimal overhead while maintaining compatibility with popular transformer-based model families such as LLaMA and GPT-style architectures.
