Download Latest Version Release v0.6.15 source code.zip (7.3 MB)
Email in envelope

Get an email when there's a new version of FlashInfer

Home / v0.6.15
Name Modified Size InfoDownloads / Week
Parent folder
flashinfer_jit_cache-0.6.15+cu130-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-17 1.8 GB
0
flashinfer_jit_cache-0.6.15+cu130-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-17 1.6 GB
1 1 weekly downloads
flashinfer_jit_cache-0.6.15+cu129-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-17 2.0 GB
0
flashinfer_jit_cache-0.6.15+cu129-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-17 2.1 GB
0
flashinfer_jit_cache-0.6.15+cu128-cp39-abi3-manylinux_2_28_aarch64.whl 2026-07-17 1.3 GB
0
flashinfer_jit_cache-0.6.15+cu128-cp39-abi3-manylinux_2_28_x86_64.whl 2026-07-17 1.3 GB
0
flashinfer_cubin-0.6.15-py3-none-any.whl 2026-07-17 611.7 MB
0
flashinfer_python-0.6.15-py3-none-any.whl 2026-07-17 14.9 MB
0
flashinfer_python-0.6.15.tar.gz 2026-07-17 10.3 MB
0
README.md 2026-07-16 12.6 kB
0
Release v0.6.15 source code.tar.gz 2026-07-16 5.5 MB
0
Release v0.6.15 source code.zip 2026-07-16 7.3 MB
0
Totals: 12 Items   10.8 GB 1

Highlights

This release ships Expert Parallelism (moe_ep) in the default install and extends the TRTLLM-GEN MoE stack for large-model serving. It broadens Blackwell model coverage — Gemma 4 / MiniMax-M3 MoE on consumer / DGX-Spark SM12x, DeepSeek-class MLA decode on B300, and context-parallel GDN on SM120 — and adds Video Sparse Attention plus CUDA-graph-safe FP8 all-reduce fusion for distributed inference.

Unified MoE API with Expert Parallelism (Experimental API to try!)

FlashInfer's unified MoE compute API is now wired into expert parallelism. A new flashinfer.moe_ep.MoEEpLayer runs one MoE layer split across ranks as dispatch → per-expert grouped GEMM → combine, over pluggable transport (NCCL-EP via nccl.ep/nccl4py, and NIXL-EP). The expert GEMM reuses the unified flashinfer.fused_moe.MoELayer as a pure per-expert grouped GEMM — routing lives in dispatch/combine. moe_ep is part of the default install (CUDA 13+), with vLLM-facing APIs and checkpoint-safe MoE all-to-all graph VAs. This EP API is also the integration surface for MegaMoE kernels in upcoming releases — try it out and let us know how it works for you.

  • [#3686]
  • [#3821]
  • [#3813]
  • [#3727]
  • [#3591]

Gemma 4 and MiniMax-M3 NVFP4 MoE now run on Blackwell SM12x

Gemma 4 and MiniMax-M3 NVFP4 MoE now run on Blackwell SM12x (consumer / DGX Spark), enabled by two new NVFP4 MoE activation functions — gelu_tanh and swiglu_oai — added to the SM12x MoE path.

  • [#3744]
  • [#3737]

TRTLLM-GEN MoE adds shared experts, DeepSeek-V4 routing, and low-latency FP8

TRTLLM-GEN MoE gains shared-expert fusion for FP8 paths, hash-based DeepSeek-V4 routing (hash_topk), and a fused FP8 blockwise megakernel that cuts latency for small batches (BS ≤ 8).

  • [#2625]
  • [#3561]
  • [#3424]

Video Sparse Attention and DeepSeek MLA decode on Blackwell B300

Video Sparse Attention (VSA) is now integrated into the block-sparse attention API, bringing efficient long-context attention for video diffusion models to FlashInfer. DeepSeek-class MLA decode extends onto Blackwell B300 (SM103) with cluster-aware CUTLASS split_kv, and CuTe-DSL GQA decode adds sliding-window and attention-sink masking for newer attention variants.

  • [#3250]
  • [#3888]
  • [#3794]

Linear attention: context-parallel GDN on SM120 and faster KDA decode

Gated delta-rule (GDN) linear attention adds SM120 context-parallel delta rule support, extending the CuTe-DSL GDN rewrite from 0.6.14 onto Blackwell SM120 prefill, alongside SM90 context-parallel prefill optimizations. KDA recurrent-decode kernels are also optimized for lower decode latency.

  • [#3659]
  • [#3788]
  • [#3766]

Distributed comm: FP8 fusion and CUDA-graph checkpoint restore

Dynamic per-token FP8 quantization fuses into allreduce + residual + RMSNorm (TRT-LLM and MNNVL backends, CUDA-graph safe). All-reduce workspaces support checkpoint_prepare / checkpoint_restore so physical backing can be released and remapped at stable VAs across serving checkpoints.

  • [#3421]
  • [#3745]

What's Changed

New Contributors

Full Changelog: https://github.com/flashinfer-ai/flashinfer/compare/v0.6.14rc1...v0.6.15

Source: README.md, updated 2026-07-16