GPUStack Files

🚀 Model Catalog Updates

LLM: Nemotron-Cascade-2-30B-A3B, NVIDIA-Nemotron-3-Nano-4B, NVIDIA-Nemotron-3-Nano-30B-A3B, NVIDIA-Nemotron-3-Super-120B-A12B, MiniMax-M2.7, Gemma-4-E2B-it, Gemma-4-E4B-it, Gemma-4-26B-A4B-it, Gemma-4-31B-it, Qwen3.6-35B-A3B, Qwen3.6-27B, GLM-5.1, Kimi-K2.6, DeepSeek-V4-Pro, DeepSeek-V4-Flash, Mistral-Medium-3.5-128B, MiMo-V2.5-Pro, MiMo-V2.5, MiniCPM-V-4.6, MiniCPM5-1B, Mellum2-12B-A2.5B-Instruct, Mellum2-12B-A2.5B-Thinking, LFM2.5-8B-A1B. Image: BAGEL-7B-MoT, GLM-Image, FLUX.2-dev, NextStep-1.1, ERNIE-Image. Audio: Voxtral-4B-TTS-2603, Fun-CosyVoice3-0.5B-2512.

✨ New Features & Enhancements

GPU Instance Service

• GPU Instance Management: Provision isolated GPU instances on demand with configurable GPU type and count, accessible via SSH (#4784).
• Support viewing GPU instance usage (#5297).

Kubernetes & Installation

• Helm Chart installation for production-grade Kubernetes deployment (#2291).
• Custom volume mounts in the Kubernetes cluster wizard (#4775).
• Cluster-level Kubernetes deployment values for worker manifests (#5379).
• Documentation for deploying GPUStack server on Kubernetes (#3669).

Database

• openGauss database support (#5041).
• Verified OceanBase database compatibility (#5123).

API Key & Usage

• Token usage per API key (#2899).
• Allow users to view their own token usage (#4881).
• Enhanced API Key management (#5083).
• Allow specifying a custom API key instead of an auto-generated one (#4100).
• Display partial characters of all API keys for identification (#2825).

Inference & Serving

• Multi-LoRA adapter support in vLLM (#2626).
• Model instance liveness probe / periodic health check (#3189).
• Automatic failover when a node stays offline (#4975).
• Support one-way (worker → server only) network reachability (#4131).
• URL-based routing with custom aliases for Generic Proxy (e.g. /model/route/{alias}) (#3515).
• Support /v1/score and /score endpoints for rerank models (#4932).
• Support the /pooling endpoint (#5052).
• Support vLLM MP distributed mode (Rayless mode) (#5037).
• Persist vLLM compilation cache directory ~/.cache/vllm/torch_compile_cache to speed up restarts (#5175).
• Expose prefix-cache hit tokens in usage for vLLM models via the OpenAI-compatible API (#5189).
• Synchronize monitoring metrics with upstream (vLLM, SGLang, MindIE) (#5110).
• Expose restart_count and latest_restart_time as metrics (#4371).
• Auto worker/GPU assignment now prefers nodes where the model data lives (#3852).
• Support viewing Ray or subordinate worker logs for multi-node distributed inference in the UI (#4158).
• Support viewing previous model instance logs (#2177).
• Optimize the error prompt and compatibility instructions for DeepSeek-V4 deployment (#5388).
• Improve compatibility checks for --port, --api-key, and --served-model-name when deploying models (#4237).
• Update huggingface_hub (#4006).

Accelerator & Backend

• Support DTK 26.04 for Hygon DCU detection (#5334).
• Include Iluvatar Corex Container Toolkit in the prerequisites check (#5264).
• Default to the latest backend version when runtime version cannot be detected (#5191).
• Custom backend parameter suggestion and auto-completion (#4745).

UI/UX

• Redesigned dashboard (#5246).
• Add server/worker version mismatch check to avoid runtime errors (#4790).
• Worker and GPU view enhancements (#1007).
• Display GPUStack version on the workers page (#5038), GPUStack version and GPU driver version in worker detail (#4845), and node version on the node page (#4843).
• Improve TTS Playground UI validation for Qwen3-TTS (#5077).
• Model Comparison: add a checkbox to sync system prompts across models (#4746).
• Address inference backend UX issues and surface previously missing configuration fields in the UI (#4717).
• Fix model deployment source display inconsistency between ModelScope and local file (#4742).
• Suppress metrics access logs (#3933).
• Optimize duplicate benchmark log messages (#4497) and the export of multiple benchmarks (#4487).
• Skip default admin auto-creation when admin users already exist after master restart (#4779).
• Disallow multiple backend parameters on a single line to prevent parsing errors (#5200).

🐛 Bug Fixes

• Fixed vLLM base image failing to start after installing packages via PYPI_PACKAGES_INSTALL (#4395).
• Fixed GPUStack hangs during compatibility check (#3934).
• Fixed server memory leak (#5073).
• Fixed GPUStack Docker container restarting 5 times a day (#5071).
• Fixed truststore + OpenSSL 3.0–3.3 interaction that caused TLS handshakes to degrade quadratically over time, leading to workers stuck in NOT_READY on TLS-enabled deployments (#5374).
• Fixed SSO single sign-on issue (#5356).
• Fixed API keys of deactivated users still working (#5414).
• Fixed updated access permission for specified models on an API key not taking effect (#5273).
• Fixed public access setting of a model not taking effect (#5168).
• Fixed missing runtime parameter in Kubernetes cluster registration URL for Hygon and T-Head (#5333).
• Fixed failure to fetch model and test model for the Claude provider (#5282).
• Fixed OpenAI provider testing error with gpt5.x models (#5011).
• Fixed OpenRouter API error when stream=false (#5053).
• Fixed double claude→openai header transformation when using one GPUStack as a backend provider for another (#4890).
• Fixed Generic Proxy not supporting URL params (#5221).
• Fixed multi-replica deployment only returning a single replica (#5158).
• Fixed distributed inference Ray join failure on multi-node deployments (#5181).
• Fixed assigned VRAM not matching actual VRAM usage in distributed deployments (#4210).
• Fixed VLLMResourceFitSelector incorrectly reporting 0.0 GiB allocatable VRAM (#5207).
• Fixed incorrect token statistics for the reranker interface (#5176).
• Fixed High TTFT in the first request after deploying small vLLM models such as Qwen3.5-0.8B (#4969).
• Fixed restart_count not increasing for model instances (#4369).
• Fixed incorrect port assignment after creating and deleting models frequently (#4302).
• Fixed escape character error in model startup (#5209).
• Fixed Hygon DCU (K100 AI) detection failure (#5170).
• Fixed "Too Many Open Files" error causing the worker to become not ready (#5259).
• Fixed benchmark pods failing in Kubernetes with hostNetwork due to missing dnsPolicy (#5133).
• Fixed benchmark creation auto-selecting pending or starting instances (#4999).
• Fixed llama.cpp backend looking for mmproj in a subfolder instead of the repo root for multimodal GGUF models (#5108).
• Fixed Qwen3-TTS issues on Ascend (#5020, #5009, #5010).
• Fixed Python client example for image editing (#5017).
• Fixed token-usage plugin failing when connected to Codex (#5137).
• Fixed HuggingFace download stalling at 98%–99% when HF_XET is enabled (#2604).
• Fixed voice interface sending an incorrect request with no error prompt (#5165).
• Fixed Hide "Allow Distributed Inference Across Workers" option being shown for custom backends (#5193).
• Fixed pagination not persisting the page-size selection (#4627).

🔧 Built-in Inference Backend Updates

New Arrival

• CANN 9.0
• vLLM 0.20.2 (310P / 910B / A3)
• CANN 8.5
• vLLM 0.18.0 / 0.17.0 (310P / 910B / A3)
• SGLang 0.5.12.post1 (910B / A3)
• CUDA 13.0
• vLLM 0.22.1 / 0.21.0 / 0.20.2 / 0.19.1 / 0.18.1
• SGLang 0.5.12.post1
• CUDA 12.9
• vLLM 0.22.1 / 0.21.0 / 0.20.2 / 0.19.1 / 0.18.1
• SGLang 0.5.12.post1
• ROCm 7.2
• vLLM 0.22.1 / 0.21.0 / 0.20.2 / 0.19.1
• SGLang 0.5.12.post1
• ROCm 7.0
• vLLM 0.18.1

Deprecated

• CUDA 12.6
• vLLM 0.15.1 / 0.14.1 / 0.13.0 / 0.12.0 / 0.11.2 / 0.10.2
• VoxBox 0.0.21
• ROCm 6.4
• vLLM 0.16.0 / 0.15.1 / 0.14.1 / 0.13.0 / 0.12.0 / 0.11.2 / 0.10.2
• SGLang 0.5.8.post1 / 0.5.7 / 0.5.6.post2 / 0.5.5.post3