What's new in 3.0.0 (2026-07-18)

These are the changes in inference v3.0.0.

New features

• FEAT: Switch default frontend to Next.js app by @OliverBryant in https://github.com/xorbitsai/inference/pull/5111
• feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in https://github.com/xorbitsai/inference/pull/5143
• feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in https://github.com/xorbitsai/inference/pull/5149
• feat(embedding): support BCE embeddings with vLLM by @amumu96 in https://github.com/xorbitsai/inference/pull/5158
• Feat: (UI) Added Security Settings Page、Audit Center Page by @maoyuehui in https://github.com/xorbitsai/inference/pull/5153
• feat: enable advanced database-backed authentication by default by @OliverBryant in https://github.com/xorbitsai/inference/pull/5144
• FEAT: production-ready Docker Compose deployment with offline private PyPI profile by @OliverBryant in https://github.com/xorbitsai/inference/pull/5174
• feat(auth): consolidate to database-backed auth and add offline password reset by @OliverBryant in https://github.com/xorbitsai/inference/pull/5180
• FEAT(webui): show per-model GPU memory in Running Models by @OliverBryant in https://github.com/xorbitsai/inference/pull/5186
• feat(llama.cpp): install GPU xllamacpp wheel from CUDA-matched index by @OliverBryant in https://github.com/xorbitsai/inference/pull/5185
• FEAT: prebuilt pypiserver mirror image as the offline compose package source by @qinrui777 in https://github.com/xorbitsai/inference/pull/5048
• FEAT: slim GPU Docker image without pre-installed inference engines by @OliverBryant in https://github.com/xorbitsai/inference/pull/5188

Enhancements

• ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5142
• ENH: update models JSON [embedding] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5159
• ENH: update 2 models JSON ("CogVideoX-2b", "HunyuanVideo") by @yiboyasss in https://github.com/xorbitsai/inference/pull/5164
• ENH: update model "CogVideoX-5b" JSON by @yiboyasss in https://github.com/xorbitsai/inference/pull/5165
• ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5171
• ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5192
• ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5197
• ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5199
• bld: Fix GitHub gpu ci by @OliverBryant in https://github.com/xorbitsai/inference/pull/5107
• bld: Narrow GPU CI triggers and drop always-skipped tests by @OliverBryant in https://github.com/xorbitsai/inference/pull/5182

Bug fixes

• fix(auth): advanced-auth live-read via DB-current permissions by @m199369309 in https://github.com/xorbitsai/inference/pull/5135
• fix(vllm): flashinfer AOT post-install hook for sm_120 Blackwell by @m199369309 in https://github.com/xorbitsai/inference/pull/5137
• fix(audio): bump funasr pin for Fun-ASR-Nano models to ~=1.3 by @m199369309 in https://github.com/xorbitsai/inference/pull/5140
• fix(frontend): route params via usePathname(); remove Gradio, add LaTeX rendering by @OliverBryant in https://github.com/xorbitsai/inference/pull/5146
• fix: replace Korean text with Japanese in ja locale launchModel keys by @kejhz653-stack in https://github.com/xorbitsai/inference/pull/5154
• fix(embedding): implement truncate_prompt_tokens, OOM eviction, and wait_for_load after recovery by @m199369309 in https://github.com/xorbitsai/inference/pull/5151
• fix(tool_parser): fix deepsee-v4 stream tool calls error by @llyycchhee in https://github.com/xorbitsai/inference/pull/5155
• fix(worker): use os.replace for atomic launch_args persist on Windows (#5161) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5162
• fix(supervisor): resolve comma-separated worker_ip in sharded launch (#5138) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5160
• fix(llama.cpp): resolve mmproj against the model directory so image input works by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5170
• fix(oauth2): revoke refresh tokens on password reset by @OliverBryant in https://github.com/xorbitsai/inference/pull/5177
• Fix: (UI) copyToClipboard 、fix the locales Running Model Page by @maoyuehui in https://github.com/xorbitsai/inference/pull/5175
• fix(core): free GPU memory when stopping pytorch rerank models by @OliverBryant in https://github.com/xorbitsai/inference/pull/5173
• BUG: sglang venv inherits incompatible parent transformers under skip_installed by @OliverBryant in https://github.com/xorbitsai/inference/pull/5181
• fix: honor virtualenv exemption in engine matching for multimodal LLM, embedding and rerank by @OliverBryant in https://github.com/xorbitsai/inference/pull/5179
• fix(image/ocr): import torchvision lazily in deepseek_ocr (#5189) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5191
• fix(worker): create model subpool after virtualenv is installed by @kejhz653-stack in https://github.com/xorbitsai/inference/pull/5196
• fix: pass torch version to offline package builds by @qinxuye in https://github.com/xorbitsai/inference/pull/5200

Documentation

• doc: update v2.12.0 release docs by @qinxuye in https://github.com/xorbitsai/inference/pull/5136
• doc: xinference release notes links by @OliverBryant in https://github.com/xorbitsai/inference/pull/5139
• DOC: document Xinference 3.0 migration and breaking changes by @OliverBryant in https://github.com/xorbitsai/inference/pull/5194

Others

• ci: narrow GPU test group detection to paths that actually affect GPU tests by @OliverBryant in https://github.com/xorbitsai/inference/pull/5148
• refactor: move XINFERENCE_MODEL_ACTOR_AUTO_RECOVER_LIMIT to constants.py by @m199369309 in https://github.com/xorbitsai/inference/pull/5152
• ci: fix Metal CI failures with latest MLX on macOS 26 runners by @OliverBryant in https://github.com/xorbitsai/inference/pull/5125
• docs(monitor): enhance metrics documentation with cluster label, status detail, and HTTP middleware by @m199369309 in https://github.com/xorbitsai/inference/pull/5184
• chore: remove legacy React web UI now that Next.js frontend is the default by @OliverBryant in https://github.com/xorbitsai/inference/pull/5145

Full Changelog: https://github.com/xorbitsai/inference/compare/v2.12.0...v3.0.0