LMDeploy Files

What's Changed

🚀 Features

• Integrate DeepEPv2 by @irexyc in https://github.com/InternLM/lmdeploy/pull/4783
• feat(pytorch): support Kimi K2.6 by @qescccczmr in https://github.com/InternLM/lmdeploy/pull/4846
• feat(kv_connector): support mooncake store by @caikun-pjlab in https://github.com/InternLM/lmdeploy/pull/4903

💥 Improvements

• feat(chat-completions): server-side fan-out for n>1 choices by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4841
• perf: further optimize GLM-5.2 serving by @CUHKSZzxy in https://github.com/InternLM/lmdeploy/pull/4853
• perf(cuda): use PDL for paged attention and V4 prefill by @grimoire in https://github.com/InternLM/lmdeploy/pull/4861
• [ascend] update attn op_backend by @wanfengcxz in https://github.com/InternLM/lmdeploy/pull/4900
• perf(pytorch): optimize compact blocked FP8 MoE and route preparation by @grimoire in https://github.com/InternLM/lmdeploy/pull/4857
• perf(pytorch): reduce speculative decoding pre/post-processing overhead by @grimoire in https://github.com/InternLM/lmdeploy/pull/4877
• support page size that are not power of two by @irexyc in https://github.com/InternLM/lmdeploy/pull/4854
• feat: support structural_tag response_format for turbomind and pytorch engines by @windreamer in https://github.com/InternLM/lmdeploy/pull/4906

🐞 Bug fixes

• fix(turbomind): restore FP8 weight-only fallback on pre-sm90 GPUs by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4871
• fix(vl): raise a clear error on malformed data URLs by @SuperMarioYL in https://github.com/InternLM/lmdeploy/pull/4837
• fix(api): fix reponses interface by @caikun-pjlab in https://github.com/InternLM/lmdeploy/pull/4893
• Fix/dsv4 native transformers warmup by @grimoire in https://github.com/InternLM/lmdeploy/pull/4878
• fix: support inline system messages in Anthropic API by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4882
• fix: bound DSA prefill score memory by @CUHKSZzxy in https://github.com/InternLM/lmdeploy/pull/4896
• fix(build): correct GEMM kernel archive link order by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4910
• fix: reject unavailable GLM tool calls by @CUHKSZzxy in https://github.com/InternLM/lmdeploy/pull/4901
• fix(pytorch): avoid Triton miscompile in paged attention reduction by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4920
• fix Intern-S2-Preview-FP8 convert by @irexyc in https://github.com/InternLM/lmdeploy/pull/4923

🌐 Other

• [Fix] Validate cross-file Markdown link targets by @JimmyWang0417 in https://github.com/InternLM/lmdeploy/pull/4868
• Update README to Reflect EuroSys 2027 Paper Acceptance by @Youhe-Jiang in https://github.com/InternLM/lmdeploy/pull/4891
• docs: fix grammar in README by @MarkHe1222 in https://github.com/InternLM/lmdeploy/pull/4866
• improve(autotest): trim unused model configs and gate routed_experts on yaml by @littlegy in https://github.com/InternLM/lmdeploy/pull/4885
• build: remove flashinfer from CUDA runtime requirements by @CUHKSZzxy in https://github.com/InternLM/lmdeploy/pull/4902
• [ci] add base api eval test workflow by @zhulinJulia24 in https://github.com/InternLM/lmdeploy/pull/4874
• bump version to v0.17.0 by @lvhan028 in https://github.com/InternLM/lmdeploy/pull/4914

New Contributors

• @JimmyWang0417 made their first contribution in https://github.com/InternLM/lmdeploy/pull/4868
• @Youhe-Jiang made their first contribution in https://github.com/InternLM/lmdeploy/pull/4891
• @MarkHe1222 made their first contribution in https://github.com/InternLM/lmdeploy/pull/4866
• @qescccczmr made their first contribution in https://github.com/InternLM/lmdeploy/pull/4846

Full Changelog: https://github.com/InternLM/lmdeploy/compare/v0.16.0...v0.17.0