Search Results for "decoding"
Sort By:
Showing 59 open source projects for "decoding"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
1
DeepSpec
A full-stack codebase for training and evaluating speculative decoding
...The evaluation pipeline measures speculative decoding performance across benchmark tasks such as math, coding, instruction-following, and chat-style datasets. Overall, it is useful for researchers and engineers studying faster language model inference through speculative decoding methods. -
2
DFlash
Block Diffusion for Ultra-Fast Speculative Decoding
DFlash is an open-source framework for ultra-fast speculative decoding using a lightweight block diffusion model to draft text in parallel with a target large language model, dramatically improving inference speed without sacrificing generation quality. It acts as a “drafter” that proposes likely continuations which the main model then verifies, enabling significant throughput gains compared to traditional autoregressive decoding methods that generate token by token. -
3
R-KV
Redundancy-aware KV Cache Compression for Reasoning Models
R-KV is an open-source research project that focuses on improving the efficiency of large language model inference through key-value cache compression techniques. Modern transformer models rely heavily on KV caches during autoregressive decoding, which store intermediate attention states to accelerate generation. However, these caches can consume large amounts of memory, especially in reasoning-oriented models with long context windows. R-KV introduces a method for compressing the KV cache during decoding, allowing models to maintain reasoning performance while reducing memory consumption and computational overhead. ... -
4
MTPLX
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B
MTPLX is an open-source inference engine and native macOS application for running large language models locally on Apple Silicon. It accelerates text generation through multi-token prediction (MTP) speculative decoding, allowing models to predict and verify several tokens simultaneously. The engine uses exact rejection sampling to preserve the model's intended output distribution. It supports Qwen 3.8, Qwen 3.6, Qwen 3.5, Gemma 4, and Ternary Bonsai models. Automatic hardware detection recommends models based on available unified memory. Users can install models, configure decoding, and monitor performance through its desktop application or command-line interface. ... -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
NVIDIA Model Optimizer
A unified library of SOTA model optimization techniques
Model Optimizer is a unified library that provides state-of-the-art techniques for compressing and optimizing deep learning models to improve inference efficiency and deployment performance. It brings together multiple optimization strategies such as quantization, pruning, distillation, and speculative decoding into a single cohesive framework. The library is designed to reduce model size and computational requirements while maintaining accuracy, making it particularly valuable for deploying large models in production environments. It supports a wide range of model types, including large language models, diffusion models, and vision-language models, and integrates with deployment frameworks such as TensorRT and vLLM. ... -
6
Omnilingual ASR
Omnilingual ASR Open-Source Multilingual SpeechRecognition
...It focuses on leveraging self-supervised audio pretraining and scalable fine-tuning so low-resource languages can benefit from high-resource data. The project provides data preparation pipelines, training scripts, decoding utilities, and evaluation tools so researchers can reproduce results and extend to new language sets. It emphasizes modularity: acoustic modeling, language modeling, tokenization, and decoding are separable pieces you can swap or ablate. The repo is aimed at pushing practical multilingual ASR—robust to accents, code-switching, and domain shifts—rather than language-by-language systems. ... -
7
PyAV
Pythonic bindings for FFmpeg's libraries
...The library is designed for precise media manipulation, making it suitable for applications that require frame-level access or custom processing pipelines. PyAV integrates well with other Python libraries such as NumPy and Pillow, allowing seamless data transformation and analysis. It supports decoding, encoding, filtering, and resampling operations, covering a wide range of multimedia tasks. While powerful, it requires a solid understanding of FFmpeg concepts, as it prioritizes flexibility and control over abstraction. Overall, PyAV is a robust tool for developers building advanced video and audio processing systems in Python. -
8
MiniCPM4.1
Achieving 3+ generation speedup on reasoning tasks
MiniCPM4.1 is an enhanced iteration of the MiniCPM4 architecture, introducing improvements in reasoning capabilities, inference speed, and hybrid operation modes that allow dynamic switching between deep reasoning and standard generation. It builds upon the same efficiency-focused philosophy but further optimizes decoding performance, achieving substantial speed gains in reasoning-intensive tasks while maintaining high-quality outputs. One of its key innovations is the hybrid reasoning mode, which allows developers to control whether the model engages in deeper reasoning processes or faster responses depending on the use case. The model also supports both dense and sparse attention mechanisms, enabling more efficient computation depending on the selected inference framework. ... -
9
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
DeepInfer
High-performance, large-scale inference framework for DeepSeek models
...It integrates optimized computational kernels and distributed parallelism techniques for attention and mixture-of-experts operations. An asynchronous scheduler overlaps CPU scheduling with accelerator execution to improve throughput. Speculative decoding and confidence-based verification help accelerate token generation. Additional capabilities include deterministic sampling, batch-invariant inference, and structured output generation using JSON schemas, regular expressions, or grammars. Complete production serving requires additional infrastructure components. -
11
OpenShot Video Editor
Award-Winning Open Source Video Editing Software
OpenShot Video Editor is a powerful yet very simple and easy-to-use video editor that delivers high quality video editing and animation solutions. OpenShot offers a myriad of features and capabilities, including powerful curve-based Key frame animations, 3D animated titles and effects, slow motion and time effects, audio mixing and editing, and so much more. It’s available for Linux, Mac and Windows, with a very simple and friendly interface. Start creating stunning videos quickly and easily... -
12
Tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models
...It also offers extension mechanisms so that custom encodings can be registered. Internally, it includes the core tokenizer logic (often implemented in Rust or efficient lower-level code), APIs for encoding, decoding, and counting tokens, and binding layers to Python (and sometimes other languages) for easy use. -
13
MiniCPM-V
See, hear, and interact on your local devices
...The larger model uses Lookahead Expert Scheduling to accelerate inference on memory-constrained devices. Quantized versions reduce hardware requirements, while speculative decoding further improves generation speed. The broader MiniCPM family also includes MiniCPM-o models for real-time audio, video, and speech interactions. Developers can use pretrained checkpoints, inference frameworks, and fine-tuning resources to build multimodal applications. -
14
URH (Universal Radio Hacker)
Universal Radio Hacker: Investigate Wireless Protocols Like A Boss
Universal Radio Hacker (URH) is a tool for investigating unknown wireless communication protocols. It supports signal capturing, decoding, modulation analysis, and protocol reverse engineering through an intuitive graphical interface. URH is used in research, security testing, and hobbyist electronics for understanding proprietary RF systems, IoT device communication, and embedded protocols. -
15
NanoJev
A nano replica of Jev: parallel decisions, dynamic candidates
NanoJev is an open-source 0.6B-parameter decision model created as a compact reproduction of Jev-style parallel decision making. It takes states and questions as input and returns complete probability distributions without decoding output tokens one by one. The project supports dynamic candidate sets so the available actions can change from one decision to the next. One shared checkpoint has been tested across ViZDoom aiming tasks, maze navigation, and Snake. The repository includes datasets, configurations, research results, demonstrations, and an end-to-end training pipeline. ... -
16
LinuxPlay
An open-source, ultra-low-latency remote desktop for Linux hosts
LinuxPlay is a multimedia playback project designed to provide a lightweight media player for Linux environments with support for common audio and video formats. It focuses on implementing playback functionality using FFmpeg and native Linux APIs to ensure efficient decoding and rendering. The project demonstrates how to build a media player from scratch, including handling audio-video synchronization and playback controls. It supports local file playback and may include streaming capabilities depending on configuration. LinuxPlay is structured as both a functional tool and a learning resource for developers interested in multimedia systems. ... -
17
Requests
A simple, yet elegant, HTTP library.
Requests is the de facto HTTP library for Python—simple, elegant, and human-friendly. It wraps urllib3 to provide intuitive methods for sending HTTP/1.1 requests, handling sessions, cookies, redirects, authentication, proxies, and more. -
18
SGLang
SGLang is a fast serving framework for large language models
SGLang is a fast serving framework for large language models and vision language models. It makes your interaction with models faster and more controllable by co-designing the backend runtime and frontend language. -
19
FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs
...The platform enables developers to deploy trained models quickly using optimized inference pipelines that support GPUs, specialized AI accelerators, and other hardware architectures. FastDeploy includes advanced acceleration technologies such as speculative decoding, multi-token prediction, and efficient KV cache management to improve throughput and latency during inference. It also offers compatibility with OpenAI-style APIs and vLLM-like interfaces, allowing developers to integrate deployed models easily into existing applications and services. -
20
Dolphin
Document Image Parsing via Heterogeneous Anchor Prompting”
Dolphin — maintained by ByteDance — is a project aimed at providing a high-performance, robust, and extensible media or multimedia framework / player infrastructure (or possibly a streaming media solution), intended to meet modern demands for efficiency, flexibility, and integration in media-heavy applications. It seeks to combine performant media playback or handling (audio/video decoding, streaming, buffering) with a modular, developer-friendly API that allows easy embedding into larger applications or services. Because multimedia delivery requirements vary widely (adaptive streaming, live feeds, cross-platform compatibility, custom UI, performance constraints), Dolphin aims to offer a foundation that developers can build upon or adapt to their needs. ... -
21
FISSURE
The RF and reverse engineering framework for everyone
FISSURE is an open-source radio frequency analysis and signal intelligence framework built to support software-defined radio research, wireless security experimentation, and protocol reverse engineering. The project brings together tools for capturing, inspecting, decoding, replaying, and analyzing RF signals across a wide range of wireless technologies. It is designed as a practical environment for researchers and operators who need to move from raw spectrum observation to structured investigation without stitching together too many separate utilities by hand. The platform supports workflows related to signal discovery, demodulation, packet inspection, fuzzing, and attack simulation, making it useful for both defensive research and controlled lab testing. ... -
22
Perception Models
State-of-the-art Image & Video CLIP, Multimodal Large Language Models
Perception Models is a state-of-the-art framework developed by Facebook Research for advanced image and video perception tasks. It introduces two primary components: the Perception Encoder (PE) for visual feature extraction and the Perception Language Model (PLM) for multimodal decoding and reasoning. The PE module is a family of vision encoders designed to excel in image and video understanding, surpassing models like SigLIP2, InternVideo2, and DINOv2 across multiple benchmarks. Meanwhile, PLM integrates with PE to power vision-language modeling, achieving results competitive with leading multimodal systems such as QwenVL2.5 and InternVL3, all while being fully reproducible with open data. ... -
23
LTX-Video
Official repository for LTX-Video
LTX-Video is a sophisticated multimedia processing framework from Lightricks designed to handle high-quality video editing, compositing, and transformation tasks with performance and scalability. It provides runtime components that efficiently decode, encode, and manipulate video streams, frame buffers, and audio tracks while exposing a rich API for building customized editing features like transitions, effects, color grading, and keyframe automation. The toolkit is built with both real-time... -
24
HTTPX
A next generation HTTP client for Python
...International domains and URLs, keep-alive and connection pooling, sessions with cookie persistence, browser-style SSL verification. Basic/digest authentication, elegant key/value cookies, automatic decompression. Automatic content decoding, unicode response bodies, multipart file uploads, HTTP(S) proxy support. Connection timeouts, streaming downloads, .netrc support, and chunked requests. For more advanced topics, see the Advanced Usage section, the async support section, or the HTTP/2 section. The Developer Interface provides a comprehensive API reference. -
25
Ciphey
Decrypt encryptions without knowing the key or cipher
Fully automated decryption/decoding/cracking tool using natural language processing & artificial intelligence, along with some common sense. You don't know, you just know it's possibly encrypted. Ciphey will figure it out for you. Ciphey can solve most things in 3 seconds or less. Ciphey aims to be a tool to automate a lot of decryptions & decodings such as multiple base encodings, classical ciphers, hashes or more advanced cryptography.
