Download Latest Version Release v0.6.15 source code.zip (7.3 MB)
Email in envelope

Get an email when there's a new version of FlashInfer

Home / v0.6.14
Name Modified Size InfoDownloads / Week
Parent folder
flashinfer_jit_cache-0.6.14+cu130-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-02 1.8 GB
0
flashinfer_jit_cache-0.6.14+cu130-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-02 1.6 GB
0
flashinfer_jit_cache-0.6.14+cu129-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-02 2.0 GB
0
flashinfer_jit_cache-0.6.14+cu129-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-02 2.0 GB
0
flashinfer_jit_cache-0.6.14+cu128-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-02 1.3 GB
0
flashinfer_jit_cache-0.6.14+cu128-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-02 1.3 GB
0
flashinfer_cubin-0.6.14-py3-none-any.whl 2026-07-02 457.9 MB
0
flashinfer_python-0.6.14-py3-none-any.whl 2026-07-02 14.6 MB
0
flashinfer_python-0.6.14.tar.gz 2026-07-02 9.9 MB
0
README.md 2026-07-01 16.5 kB
0
Release v0.6.14 source code.tar.gz 2026-07-01 5.0 MB
0
Release v0.6.14 source code.zip 2026-07-01 6.7 MB
0
Totals: 12 Items   10.5 GB 0

Manual Intervention Needed ⚠️

Please modify your install instructions as you bump the version to 0.6.14.

pip install flashinfer-python

pip install flashinfer-cubin --index-url https://flashinfer.ai/whl     # this is the difference

pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu129
# OR 
pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu130

See also https://github.com/flashinfer-ai/flashinfer/issues/3808

Highlights

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

Gemma 4 and W4A16 on Blackwell RTX Pro and DGX Spark

Blackwell SM12x parts (RTX PRO 6000, DGX Spark / GB10) previously trailed Blackwell GB200 NVL72 (and B300 SM103) on attention shape and quantization coverage. This release closes the gap for Gemma 4 and weight-only-quantized inference: head_dim=512 attention for Gemma 4's global layers now runs on SM120/121, the FMHAv2 prefill path gains head_dim=256/512 and sliding-window masking on SM120, the new mm_bf16_fp4 W4A16 GEMM is tuned for DGX Spark (completing W4A16 across both dense GEMM and MoE), and gated tanh-GELU brings Gemma 4 MoE onto the CUTLASS backend.

GDN / gated delta rule: CuTe-DSL overhaul (Qwen3.5/3.6)

The gated-delta-rule kernels behind the Qwen3.5/3.6 family were rewritten from CUTLASS C++ to CuTe-DSL (https://github.com/flashinfer-ai/flashinfer/issues/3491). The rewrite eliminates the C++ JIT compilation pain reported by customers and establishes the base for context-parallel delta-rule kernels — covering SM90 prefill and its context-parallel variant, SM120 prefill, and a ~20–25% GDN prefill speedup from mainloop efficiency work.

GDN production-ready in vLLM

Two gaps blocked GDN serving in vLLM (https://github.com/flashinfer-ai/flashinfer/issues/3602); both are now resolved. GDN kernels are compilation batch-size agnostic, so a single compiled cubin is reused across batch shapes instead of recompiling on every new batch size at inference time, and a new BF16 state recovery/decode kernel writes SSM state into preallocated space to supply the MTP-compatible spec-decode path vLLM needs.

DeepSeek-class sparse MLA on Blackwell, FP8 KV on Hopper

New sparse-MLA paged-attention kernels extend the DeepSeek-V4 (d_qk=512) and DeepSeek-V3.2 / GLM-5.1 (d_qk=576) families onto SM120/121 through the existing flashinfer.mla APIs, with DSv4 coverage broadened to 8/16/32 head counts. On Hopper SM90, native FP8 KV cache support eliminates SGLang's per-layer cast workaround, saving an HBM round-trip per layer on DeepSeek-V3/V4 while staying bit-identical to the BF16 path.

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

What's Changed

New Contributors

Full Changelog: https://github.com/flashinfer-ai/flashinfer/compare/v0.6.13rc2...v0.6.14

Source: README.md, updated 2026-07-01