DownloadLing-3.0-tiny is inclusionAI’s lightweight hybrid reasoning Mixture-of-Experts model, designed to provide capable reasoning and agentic performance at low inference cost. It contains 7.9B total parameters while activating only 1.3B per token, using a hybrid architecture that alternates Kimi Delta Attention and Multi-Head Latent Attention with a sparse 128-expert MoE. The model supports both fast responses and configurable multi-step thinking, covering general agents, coding, mathematics, scientific reasoning, and instruction following. It is specifically optimized for local and resource-constrained deployment and has been validated on NVIDIA DGX Spark, Apple Silicon MacBooks, and Mac mini systems. FP8 testing reached around 100–105 tokens/s on DGX Spark and 86–90 tokens/s on an M4 Pro MacBook. BF16, FP8, and INT4 weights are available, while deployment options include SGLang, vLLM, and experimental Ollama support on Apple Silicon.
Features
- 7.9B total parameters with only 1.3B activated per token
- Hybrid KDA and MLA attention architecture
- Sparse MoE with 128 routed experts
- Configurable fast-response and multi-step thinking modes
- Agentic, coding, mathematical, and scientific reasoning
- Optimized for local and resource-constrained hardware
- BF16, FP8, and INT4 model weights
- Supports SGLang, vLLM, and Apple Silicon deployment
Project Samples
