Download
MTPLX is an open-source inference engine and native macOS application for running large language models locally on Apple Silicon. It accelerates text generation through multi-token prediction (MTP) speculative decoding, allowing models to predict and verify several tokens simultaneously. The engine uses exact rejection sampling to preserve the model's intended output distribution. It supports Qwen 3.8, Qwen 3.6, Qwen 3.5, Gemma 4, and Ternary Bonsai models. Automatic hardware detection recommends models based on available unified memory. Users can install models, configure decoding, and monitor performance through its desktop application or command-line interface. An OpenAI- and Anthropic-compatible local server enables integration with AI coding assistants and other applications. Its optimization techniques improve generation speed without requiring a separate draft model.
Features
- Native multi-token speculative decoding optimized for Apple Silicon processors
- Support for Qwen, Gemma, and Ternary Bonsai language model families
- Exact rejection sampling with residual correction at native sampling temperatures
- Automatic hardware detection, model recommendations, and decoding depth optimization
- OpenAI- and Anthropic-compatible local inference server for AI application integration
- Native macOS application and command-line interface with automated model downloads
Project Samples
