Download Latest Version Release v0.6.18.post1 source code.zip (11.9 MB) Google Add to Preferred Sources
Home / v0.6.18
Name Modified Size InfoDownloads / Week
Parent folder
flashinfer_jit_cache-0.6.18+cu134-cp39-abi3-manylinux_2_28_aarch64.whl 2026-08-29 1.4 GB
0
flashinfer_jit_cache-0.6.18+cu134-cp39-abi3-manylinux_2_28_x86_64.whl 2026-08-29 1.2 GB
0
flashinfer_jit_cache-0.6.18+cu130-cp39-abi3-manylinux_2_28_aarch64.whl 2026-08-29 1.1 GB
0
flashinfer_jit_cache-0.6.18+cu130-cp39-abi3-manylinux_2_28_x86_64.whl 2026-08-29 1.0 GB
1 1 weekly downloads
flashinfer_jit_cache-0.6.18+cu129-cp39-abi3-manylinux_2_28_aarch64.whl 2026-08-29 1.0 GB
0
flashinfer_jit_cache-0.6.18+cu129-cp39-abi3-manylinux_2_28_x86_64.whl 2026-08-29 1.0 GB
0
flashinfer_cubin-0.6.18-py3-none-any.whl 2026-08-29 1.6 GB
9 9 weekly downloads
flashinfer_python-0.6.18-py3-none-any.whl 2026-08-29 18.3 MB
0
flashinfer_python-0.6.18.tar.gz 2026-08-29 13.3 MB
0
README.md 2026-08-28 30.8 kB
0
Release v0.6.18 source code.tar.gz 2026-08-28 9.5 MB
0
Release v0.6.18 source code.zip 2026-08-28 11.9 MB
0
Totals: 12 Items   8.4 GB 10

These highlights are also published at flashinfer.ai/releases.

v0.6.18 Highlights

This release completes NVIDIA Rubin (SM107) support, brings whole-layer expert-parallel MoE to Hopper, adds decode paths for DeepSeek-V4 sparse attention and Kimi K3 linear attention, and broadens low-precision MoE coverage with MXFP4 on Blackwell RTX PRO and DGX Spark and weight-only W4A16 on B200 and B300.

SM107 (Rubin) support

FlashInfer 0.6.18 completes support for NVIDIA Rubin (SM107), begun in 0.6.16. Rubin devices now dispatch through the same unified APIs as Blackwell — attention, GEMM, MoE, and quantization.

Attention gains trtllm-gen FMHA for SM107, including sparse compression and FP16 softmax. The PrimTS attention path accepts Rubin as well. On the GEMM side, batched and low-latency GEMM both run on Rubin, and CUTLASS NVFP4 SVDQuant is enabled. A CuTe-DSL kernel family specialized for Rubin ships alongside them. trtllm-gen MoE now selects valid tactics on Rubin, and router GEMM and topk_varlen accept SM107.

Expert-parallel MoE runs whole-layer on Hopper

The moe_ep mega-kernel stack was Blackwell-only. Two Hopper FP8 backends, Sm90PullFp8MegaMoeConfig and Sm90PushFp8MegaMoeConfig, now let Hopper deployments run dispatch, FC1, SwiGLU, FC2, and combine as one fused layer behind the existing MoEEpLayer, instead of composing an NCCL all-to-all with a local fused-MoE operator. The pull backend reaches 562 TFLOPS/rank at a 384-expert DeepSeek-class geometry. The push backend supports CUDA Graph capture, and its opt-in fused FC1 epilogue drops an approximately 1 GiB per-rank activation buffer at the DeepSeek-V3 EP8 shape. Its grouped GEMM requires CUDA Toolkit 12.8 or newer.

HCA decode backend and top-K selection for DeepSeek-V4

Sparse attention picks the top-K KV positions per request on every decode step, then attends over the compressed cache; both halves now have dedicated paths. FP8 Heavily Compressed Attention (HCA) arrives for SM100/SM103 via trtllm_batch_decode_sparse_mla_dsv4(..., backend="cute-dsl"), taking arbitrary sliding-window row order including ring rotation and wraparound while keeping the compressed cache paged. The new flashinfer.top_k_varlen handles ragged batches through a Blackwell radix kernel, a guess-verify-refine kernel that warm-starts from the previous step's indices, and a CUTLASS fallback for any GPU. SM120/121 also picks up top-k 192 and 256.

Kimi K3 decode fuses into one Blackwell kernel

flashinfer.fused_kda_decode folds Kimi K3's width-four depthwise causal convolution, SiLU, recurrent Kimi Delta Attention update, and gated RMSNorm into a single SM100 launch, covering the production head_dim 128 shapes at 12, 24, 48, and 96 heads and updating the convolution cache and FP32 state in place. On B200 under CUDA Graphs it is 1.33x the vLLM fused kernel at one row (table geomean 1.13x). A T=1 fast path inside recurrent_kda and shared SM100-family recurrent kernels round out the KDA stack.

MXFP4 MoE and video sparse attention on Blackwell RTX PRO

MXFP4 checkpoints run natively on SM120/121: b12x_fused_moe and B12xMoEWrapper accept quant_mode="mxfp4" across the existing fused schedules including CUDA Graph reuse, the b12x dense GEMM gains the matching path, and on an RTX PRO 6000 Blackwell Server Edition MXFP4 tracks NVFP4 against a strict quantized reference. Video Sparse Attention, until now datacenter-Blackwell only, reaches these parts through a vsa_sm120_blk64 backend on BlockSparseAttentionWrapper. Gemma 4 gains asymmetric VO-split NVFP4 paged prefill on SM120/121.

W4A16 MoE and dense GEMM extend to B200 and B300

Weight-only NVFP4 against BF16 activations, which 0.6.14 shipped for SM12x, now runs on the SM100 family: CuteDslMoEWrapper and cute_dsl_fused_moe_nvfp4 accept quant_mode="w4a16", decoding weights to BF16 inside the kernel so no separate activation-quantization or repack launch is needed, and mm_bf16_fp4 gains a dedicated SM100/SM103 kernel. Consuming BF16 directly pays off where MoE decode is memory-bound: at a DeepSeek EP8 shape on B200, W4A16 is 1.50x the W4A4 baseline at one token; W4A4 still wins large-batch prefill. GeGLU-tanh and SiTU are supported.

Unified MoE API adds shared experts, MXINT4, and CUTLASS runners

Shared experts now work through the unified API rather than low-level kernel entry points, via ExpertConfig.num_fused_shared_experts on the block-FP8 and FP4 runners, completing in the unified API what 0.6.15 and 0.6.17 added for FP8 and FP4. The API also gains MXINT4, CUTLASS BF16 and W4A16 runners on SM90, packed per-tensor FP8 routing, BF16 FromLogits routing, and TopKSigmoid.

Fused MNNVL all-reduce tail for tensor-parallel MoE

allreduce_fusion gains a BF16 Blackwell CuTe-DSL backend for the latency-critical tail of tensor-parallel MoE layers, fusing all-reduce, residual add, and RMSNorm, optionally preceded by MoE finalize and shared-expert add, over MNNVL/NVLink multicast. One backend spans decode to prefill by switching protocol with token count, with initial profiles targeting GB300 on TP8 and TP16.

Smaller JIT-cache wheels; SM75 and single-request FA2 are JIT-only

The flashinfer-jit-cache wheels no longer ship precompiled kernels for SM75 (Turing). Those GPUs still run; the kernels compile on first use. CUDA 13 AArch64 wheels also drop native SM121a cubins (DGX Spark keeps running via SM120 family cubins), and the single-request single_decode_with_kv_cache / single_prefill_with_kv_cache FA2 modules are no longer AOT-prebuilt — those APIs still JIT. Fatbins use size-oriented compression.

0.6.17 0.6.18
cu129 x86_64 1.94 GB 1.02 GB
cu130 x86_64 1.51 GB 1.02 GB
cu130 aarch64 1.69 GB 1.13 GB

What's Changed

New Contributors

Full Changelog: https://github.com/flashinfer-ai/flashinfer/compare/v0.6.17rc5...v0.6.18

Source: README.md, updated 2026-08-28