Make your agents learn from experience
Power CLI and Workflow manager for LLMs (core package)
A Simple and Universal Swarm Intelligence Engine
TokenSpeed is a speed-of-light LLM inference engine
Universal LLM Deployment Engine with ML Compilation
A tension reasoning engine over 131 S-class problems
Alibaba's high-performance LLM inference engine for diverse apps
SQL-Driven RAG Engine
Jlama is a modern LLM inference engine for Java
A high-performance inference engine for AI models
A Next-Generation Training Engine Built for Ultra-Large MoE Models
A high-throughput and memory-efficient inference and serving engine
AI-Powered Data Processing: Use LOTUS to process all of your datasets
950 line, minimal, extensible LLM inference engine built from scratch
A lightweight vLLM implementation built from scratch
Query anything (GitHub, Notion, +40 more) with SQL and let LLMs
AI search engine - self-host with local or cloud LLMs
Mooncake is the serving platform for Kimi
LLM inference in C/C++
Emscripten: An LLVM-to-WebAssembly Compiler
A @ClickHouse fork that supports high-performance vector search
local-first semantic code search engine
Fast Multimodal LLM on Mobile Devices
High-performance inference framework for large language models
Fast, flexible LLM inference