Download Latest Version
v1.8.0 source code.tar.gz (32.9 MB)
Get an email when there's a new version of Xorbits Inference
| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| README.md | 2025-07-20 | 2.5 kB | |
| v1.8.0 source code.tar.gz | 2025-07-20 | 32.9 MB | |
| v1.8.0 source code.zip | 2025-07-20 | 33.7 MB | |
| Totals: 3 Items | 66.5 MB | 0 | |
What's new in 1.8.0 (2025-07-20)
These are the changes in inference v1.8.0.
New features
- FEAT: Embedding support llama.cpp backend by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3730
- FEAT: non-stream tool calling for sglang by @aniya105 in https://github.com/xorbitsai/inference/pull/3760
- FEAT: support migrate from v1 to v2 for custom models by @qinxuye in https://github.com/xorbitsai/inference/pull/3810
- FEAT: FLUX.1-Kontext-dev by @qinxuye in https://github.com/xorbitsai/inference/pull/3728
- FEAT: support ERNIE 4.5 by @qinxuye in https://github.com/xorbitsai/inference/pull/3812
- FEAT: [embedding] add support for jina-embeddings-v4 model by @Minamiyama in https://github.com/xorbitsai/inference/pull/3814
- FEAT: [model] support glm-4.1v-thinking by @llyycchhee in https://github.com/xorbitsai/inference/pull/3756
Enhancements
- ENH: Pin xllamacpp>=0.1.23 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3780
- ENH: add modelscope for fish speech 1.5 by @qinxuye in https://github.com/xorbitsai/inference/pull/3750
- REF: [V2 BREAK] Merge multiple JSON files into one for difference model download sources by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/3765
Bug fixes
- BUG: disable flash_attn for qwen3 embedding & rerank when no gpu available by @qinxuye in https://github.com/xorbitsai/inference/pull/3739
- BUG: Fix bugs in del async_client by @zhcn000000 in https://github.com/xorbitsai/inference/pull/3753
- BUG: add message preprocessing to ensure that content is not null by @amumu96 in https://github.com/xorbitsai/inference/pull/3791
- BUG: pre check to prevent from list index out of range for FunASR family models by @leslie2046 in https://github.com/xorbitsai/inference/pull/3809
- BUG: resolve issue where AI output was lost when no tool was selected for function call [#3767] by @aniya105 in https://github.com/xorbitsai/inference/pull/3768
- BUG: fix error in
contentoutput atreasoning_content, when usingenable_thinkinginchat_template_kwargsby @amumu96 in https://github.com/xorbitsai/inference/pull/3794
Documentation
- DOC: fix links by @qinxuye in https://github.com/xorbitsai/inference/pull/3774
- DOC: update info in docs by @qinxuye in https://github.com/xorbitsai/inference/pull/3779
- DOC: update models by @qinxuye in https://github.com/xorbitsai/inference/pull/3815
Full Changelog: https://github.com/xorbitsai/inference/compare/v1.7.1...v1.8.0