What's new in 3.1.0 (2026-07-31)

These are the changes in inference v3.1.0.

New features

• feat(ui): add SSO/OIDC login button to login page by @m199369309 in https://github.com/xorbitsai/inference/pull/5218
• feat: launch model worker multi select by @maoyuehui in https://github.com/xorbitsai/inference/pull/5222
• feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in https://github.com/xorbitsai/inference/pull/5172
• feat(webui): add auto-refresh to audit center by @m199369309 in https://github.com/xorbitsai/inference/pull/5226
• feat(webui): show API key name and model ID columns in audit center by @m199369309 in https://github.com/xorbitsai/inference/pull/5229
• feat(webui): multi-dashboard monitor center with admin config dialog by @m199369309 in https://github.com/xorbitsai/inference/pull/5225
• feat(frontend): auto-reload shell routes on export directory change by @m199369309 in https://github.com/xorbitsai/inference/pull/5243
• feat(vllm): populate logprobs/top_logprobs in completion responses by @SuperMarioYL in https://github.com/xorbitsai/inference/pull/5245
• feat(webui): support multi-node selection in log center by @m199369309 in https://github.com/xorbitsai/inference/pull/5246
• feat: speculative decoding with paired drafter checkpoints by @qinxuye in https://github.com/xorbitsai/inference/pull/5241
• feat(engine): [image, audio] sglang image vllm asr engines by @OliverBryant in https://github.com/xorbitsai/inference/pull/5121
• feat(webui): add missing permission scopes (models:register, logs:list, monitor:view) to UI by @m199369309 in https://github.com/xorbitsai/inference/pull/5254
• FEAT: [model] MiniMax-M3 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/5258

Enhancements

• ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5214
• ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5233
• ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5236
• ENH: display replica launch status in responsive grid by @leslie2046 in https://github.com/xorbitsai/inference/pull/5239
• ENH: update models JSON [audio, embedding, image, llm, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5251
• ENH: adopt xoscar 0.9.7 graceful #system_*# fallback by @OliverBryant in https://github.com/xorbitsai/inference/pull/5228
• ENH: update model "MiniMax-M3" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/5263
• BLD: build the aarch64 image from the unified slim Dockerfile by @OliverBryant in https://github.com/xorbitsai/inference/pull/5209
• BLD: improve Docker build caching by @leslie2046 in https://github.com/xorbitsai/inference/pull/5231
• BLD: require peft>=0.18.0 for transformers 5.x compatibility by @OliverBryant in https://github.com/xorbitsai/inference/pull/5260

Bug fixes

• fix(embedding/rerank): fix sentence_transformers load failures from torch-ecosystem package mismatches by @OliverBryant in https://github.com/xorbitsai/inference/pull/5210
• fix(webui): show model engine on running-model detail page by @OliverBryant in https://github.com/xorbitsai/inference/pull/5211
• fix(pypiserver): emit model source paths with forward slashes by @OliverBryant in https://github.com/xorbitsai/inference/pull/5216
• fix: (UI) Update the label of n_gpu except for LLM and Image types by @maoyuehui in https://github.com/xorbitsai/inference/pull/5217
• FIX: mitigate Slow HTTP DoS (Slowloris) by enforcing a request receipt deadline by @OliverBryant in https://github.com/xorbitsai/inference/pull/5213
• fix(api): cast replica to int to prevent TypeError on multi-replica launch by @m199369309 in https://github.com/xorbitsai/inference/pull/5220
• fix(venv): pin torch to system version to prevent torchvision ABI crash on relaunch by @OliverBryant in https://github.com/xorbitsai/inference/pull/5212
• BUG: worker keeps stale model state after a launch fails during async engine load by @OliverBryant in https://github.com/xorbitsai/inference/pull/5193
• fix(audio/indextts2): declare the missing IndexTTS2 virtualenv dependencies (#5201) by @OliverBryant in https://github.com/xorbitsai/inference/pull/5223
• fix(vllm): add #system_numpy# and pin vllm to 0.21.0 for Ornith/qwen3.5/qwen3.6 by @m199369309 in https://github.com/xorbitsai/inference/pull/5235
• fix(mlx): keep VLM generation on a dedicated thread by @qinxuye in https://github.com/xorbitsai/inference/pull/5234
• fix: disambiguate replica model UIDs with -rep{n} suffix by @Solaris-star in https://github.com/xorbitsai/inference/pull/5215
• fix: allow virtualenv packages to override system markers by @LauraGPT in https://github.com/xorbitsai/inference/pull/5204
• fix(embedding): re-normalize per-item index after batch split by @m199369309 in https://github.com/xorbitsai/inference/pull/5240
• FIX: suppress GPU warnings in CPU image by @leslie2046 in https://github.com/xorbitsai/inference/pull/5238
• fix(i18n): change SSO button text from Keycloak to generic SSO by @m199369309 in https://github.com/xorbitsai/inference/pull/5244
• bug: cover list_models tolerance for stale replica uids by @OliverBryant in https://github.com/xorbitsai/inference/pull/5168
• fix: stabilize Jina v5 and Qwen3 reranker deployment by @OliverBryant in https://github.com/xorbitsai/inference/pull/5232
• FIX: worker reconnection after supervisor restart by @leslie2046 in https://github.com/xorbitsai/inference/pull/5250
• fix(webui): improve worker node display in multi-select by @m199369309 in https://github.com/xorbitsai/inference/pull/5255
• fix(worker): skip redundant virtualenv setup when replicas share the same venv by @m199369309 in https://github.com/xorbitsai/inference/pull/5256
• fix(audio): don't reopen a NamedTemporaryFile so TTS works on Windows (#5261) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5262

Documentation

• doc: fix v3.0.0 Chinese release links by @qinxuye in https://github.com/xorbitsai/inference/pull/5205

Others

• ci: use merge-base diff for PR GPU tests by @qinxuye in https://github.com/xorbitsai/inference/pull/5224
• docs: update release notes links from xinference.io to xinference.co by @martinma51 in https://github.com/xorbitsai/inference/pull/5248
• Consolidate packaging configuration into pyproject.toml by @OliverBryant in https://github.com/xorbitsai/inference/pull/5249
• docs(i18n): update ja/ko/pt_BR release notes links to xinference.co by @martinma51 in https://github.com/xorbitsai/inference/pull/5253

New Contributors

• @Solaris-star made their first contribution in https://github.com/xorbitsai/inference/pull/5215
• @LauraGPT made their first contribution in https://github.com/xorbitsai/inference/pull/5204
• @SuperMarioYL made their first contribution in https://github.com/xorbitsai/inference/pull/5245
• @martinma51 made their first contribution in https://github.com/xorbitsai/inference/pull/5248

Full Changelog: https://github.com/xorbitsai/inference/compare/v3.0.0...v3.1.0