What's Changed
- fix(sampling): reject unsafe multi-CTA top-k launches on low-SM GPUs by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4595
- feat(cake_kda): optimize small-BH recurrent-KDA prefill by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4571
- test: enable the unified MoE fuzzer by default and prune legacy UTs by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4475
- feat(gdn): add pooled state and state checkpointing and dtype support for feature parity by @guangyunh-nv in https://github.com/flashinfer-ai/flashinfer/pull/4436
- feat(attention): add PrimTS Q64/KV256 and paged GQA block-sparse attention by @heyuhhh in https://github.com/flashinfer-ai/flashinfer/pull/4474
- add backward-compatible aliases for bsa_attn_fwd and bsa_attn_blk64_fwd by @hsr1234563 in https://github.com/flashinfer-ai/flashinfer/pull/4590
- fix(moe): restore SM12x MoE kernels broken by self-resolved helper in borrowed dense methods by @lucifer1004 in https://github.com/flashinfer-ai/flashinfer/pull/4602
- Add @Anerudhan to CODEOWNERS for core review by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4622
- fix(moe_ep): fix in_kernel_fc2_reduce livelock on zero-token launches (MXFP8 + NVFP4) by @mhoqueanik in https://github.com/flashinfer-ai/flashinfer/pull/4531
- feat(moe_ep): SM100 BF16 CuTeDSL MegaMoE kernel by @mhoqueanik in https://github.com/flashinfer-ai/flashinfer/pull/4386
- feat: CuTe DSL kernels for Rubin (SM107) and batched FP8 GEMM for Blackwell by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4526
- Add back TRTLLM Gen MoE split-K by @jiahanc in https://github.com/flashinfer-ai/flashinfer/pull/4617
- feat(kda): add CuTe DSL recurrent prefill backend by @Observer007 in https://github.com/flashinfer-ai/flashinfer/pull/4605
- feat(cake_backend): accelerate DeepSeek fused routing by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4587
- Stop legacy nightly release publishing by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4623
- feat(cake_backend): add Blackwell Router GEMM by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4594
- [feat]custom all reduce kernel by @qsang-nv in https://github.com/flashinfer-ai/flashinfer/pull/4393
- fix(attention): handle extreme negative logits in masked softmax by @shoutoutuoadi325 in https://github.com/flashinfer-ai/flashinfer/pull/4401
- feat: trtllm-gen FMHA features for sm107 (spcompress, fp16softmax) by @jimmyzho in https://github.com/flashinfer-ai/flashinfer/pull/4596
- feat(moe_ep): SM100 W4A8 (MXFP8xMXFP4) CuTeDSL split kernel backend with MXFP8 packed dispatch by @mhoqueanik in https://github.com/flashinfer-ai/flashinfer/pull/4529
- Add Qwen fused GDN decode step for sm120 by @nv-yunzheq in https://github.com/flashinfer-ai/flashinfer/pull/4481
- feat: collect a union of TEST_PATH targets in unit CI by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4641
- fix(moe): correct unified fuzzer references by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4639
- [perf] split TRT-LLM Gen routing kernels to reduce compile time by @jiahanc in https://github.com/flashinfer-ai/flashinfer/pull/4635
- feat(moe): add SiTU-GLU activation to the CUTLASS fused-MoE backend by @xuanyu-mistral in https://github.com/flashinfer-ai/flashinfer/pull/4460
- feat(cake_mamba): add Blackwell selective state update backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4616
- ci: consolidate CUDA coverage and validate candidate images by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4469
- feat(kda): add SM120a CuTe DSL prefill backend by @JimpleMa in https://github.com/flashinfer-ai/flashinfer/pull/4633
- feat(moe): standalone trtllm-gen routing op + decomposed tests/moe routing matrix by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4082
- feat(cake_vsa): add optimized SM100/SM103 block-sparse attention (VSA) by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4593
- feat(cake_kda): add paired recurrent training for SM100a and SM103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4636
- perf(gemm): optimize CuTe DSL W4A16 dense GEMM by @zianglih in https://github.com/flashinfer-ai/flashinfer/pull/4686
- fix(sm120): align MXFP8 plain tactic and FP8 moe stage policy by @CarstyYou in https://github.com/flashinfer-ai/flashinfer/pull/4660
- feat(moe): align unified MoE do_finalize behavior with flat API by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4614
- perf(gdn): reduce non-CP CuTeDSL launch overhead by @guangyunh-nv in https://github.com/flashinfer-ai/flashinfer/pull/4699
- feat(cake_mamba): add Blackwell Mamba SSDCombined by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4576
- feat: SM120 NVFP4 SVDQuant Gemm in CuteDSL by @rosenrodt in https://github.com/flashinfer-ai/flashinfer/pull/4420
- ci: disable sccache for cu134 nvcc by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4682
- Add paged MQA logits (attn_scores) kernels for Blackwell SM100 by @dhiraj113 in https://github.com/flashinfer-ai/flashinfer/pull/4365
- perf(sm120): optimize NVFP4 attention with N64 score-slot reuse by @tiffany940107 in https://github.com/flashinfer-ai/flashinfer/pull/4502
- feat(cake_msa): add Blackwell minimax sparse attention source kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4355
- perf(cake_kda): further optimize recurrent-KDA prefill on Blackwell by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4675
- feat(moe): allow B12xMoEWrapper to share pre-allocated workspaces by @lucifer1004 in https://github.com/flashinfer-ai/flashinfer/pull/4603
- (perf) add fused_GDN_step support for Qwen 3.6 35B A3B on sm120 by @nv-yunzheq in https://github.com/flashinfer-ai/flashinfer/pull/4708
- fix: SageAttention support block size doesn't divide sequence; support K-smoothing by @xrq-phys in https://github.com/flashinfer-ai/flashinfer/pull/4654
- ci: coordinate CUDA dependency policy by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4711
- perf(msa): chunked top-k and in-kernel causal offsets for the SM12x indexer by @yichengj0 in https://github.com/flashinfer-ai/flashinfer/pull/4030
- perf(cake_kda): add recurrence-piece persistent M128 prefill by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4728
- feat(moe): add remaining CUTLASS unified MoE runners by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4610
- fix: Correctly wire scale_qkvo to cute-dsl fmha backends by @xrq-phys in https://github.com/flashinfer-ai/flashinfer/pull/4665
- fix(kda): fall back to Cake when CuTe DSL predates cutlass.experimental by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4667
- fix(ci): skip source-only CUDA config test in nightlies by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4750
- perf(activation): cap act_and_mul_kernel block size for ~17-19% speedup at large hidden dims by @yekerr in https://github.com/flashinfer-ai/flashinfer/pull/4733
- fix(fmha): select CGA reduction for MLA H512 decode by @yihwang-nv in https://github.com/flashinfer-ai/flashinfer/pull/4702
- feat(norm): fused Add+RMSNorm+1x128 fp8 block-quant producer by @NVShreyas in https://github.com/flashinfer-ai/flashinfer/pull/4480
- chore(aot): exclude single prefill/decode modules from jit-cache prebuilds by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4760
- build(jit): reduce JIT-cache wheel size by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4757
- feat(moe): add unified activation parity by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4613
- Support per-token NVFP4 ReLU2 MoE by @xuantengh in https://github.com/flashinfer-ai/flashinfer/pull/4618
- feat: CUB
DeviceBatchedTopKtop-k backend with variable-length support by @NaderAlAwar in https://github.com/flashinfer-ai/flashinfer/pull/4442 - feat(cake_fmha): add native Blackwell DCP speculative decode by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4518
- fix(cute_dsl): consult the arch gate in the GEMM and GDN dispatchers by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4649
- feat(cake_activation): add fused Blackwell SwiGLU MXFP8 quantization by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4638
- Adding var-window FMHA context support for PrimsTS. by @mingxu1067 in https://github.com/flashinfer-ai/flashinfer/pull/4599
- Add bias support to cublast gemm backend and as fallback backend for cutedsl gemm backend by @jiahanc in https://github.com/flashinfer-ai/flashinfer/pull/4772
- misc: multi-arch cubins (sm100, 103, 107) in a single artifact by @jimmyzho in https://github.com/flashinfer-ai/flashinfer/pull/4648
- refactor(mla): isolate planned FA2, FA3, and CUTLASS backends by @saltyminty in https://github.com/flashinfer-ai/flashinfer/pull/4697
- Enable CuTe DSL MLA benchmarks for low head counts by @lunarz-dev in https://github.com/flashinfer-ai/flashinfer/pull/4656
- fix(gemm): never move the shared cuDNN GEMM workspace by @yanqinz2 in https://github.com/flashinfer-ai/flashinfer/pull/4666
- feat(cake_kda): add recurrent training template dispatch by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4726
- fix(norm): limit add RMSNorm FP4 launch config heuristics to SM100 and SM103 by @soodoshll in https://github.com/flashinfer-ai/flashinfer/pull/4494
- perf(cake_vsa): refresh Blackwell block-sparse WS kernel by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4804
- feat(decode): add prims-ts backend and is_causal to paged decode by @elwhyjay in https://github.com/flashinfer-ai/flashinfer/pull/4739
- [Bugfix] Skip .item() readback for trtllm_ragged_attention_deepseek during CUDA graph capture (#4609) by @zhang-keliang in https://github.com/flashinfer-ai/flashinfer/pull/4703
- feat(cake_comm): Add a Cake Blackwell all-gather matmul backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4722
- feat(cake_bgmv): add deterministic prepared Blackwell MoE backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4821
- feat(cake_backend): add grouped MXFP8 quantization by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4820
- feat(attention): add variant_owns_mask for JIT variants that own the full mask by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/4695
- feat(cake_kda): add native unbounded-softplus Kimi-Linear kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4535
- feat(cute_dsl): replace SM100 blk64 BSA C++/CUTLASS kernel with CuTe-DSL by @hsr1234563 in https://github.com/flashinfer-ai/flashinfer/pull/4612
- feat(mla): support variable-Q decode with DCP in CuTeDSL by @Observer007 in https://github.com/flashinfer-ai/flashinfer/pull/4719
- fix: support class method re-exports in API checker by @saltyminty in https://github.com/flashinfer-ai/flashinfer/pull/4717
- perf(sm12x): optimize and unify static MoE by @EricChen02 in https://github.com/flashinfer-ai/flashinfer/pull/4718
- fix(attention): make PrimTS paged block-sparse metadata live by @heyuhhh in https://github.com/flashinfer-ai/flashinfer/pull/4664
- feat(MoE): FP8 MoE per-channel quant support by @raayandhar in https://github.com/flashinfer-ai/flashinfer/pull/2809
- fix(ci): skip subprocess torch.compile tests when kernels are absent from jit-cache by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4783
- Unify MoE CuTe DSL dispatch to be dtype agnostic by @PetersonGuo in https://github.com/flashinfer-ai/flashinfer/pull/4793
- fix(cute_dsl): make the optional-dependency guard independent of cutlass by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4753
- perf(attention): cut NVFP4 KV dequant instructions on targets without native FP4/FP8 conversion by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/4746
- feat(moe): add cuTile fused MoE backend for BF16 and NVFP4 Unified MoE by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4646
- fix: skip SM107 low-latency GEMM cubins on Blackwell (#4773) by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4848
- feat(prims_ts): support no-padding MLA query rows by @PerkzZheng in https://github.com/flashinfer-ai/flashinfer/pull/4830
- Fix TRTLLM MOE per-token NVFP4 TileN 192 and update cubins by @jiahanc in https://github.com/flashinfer-ai/flashinfer/pull/4741
- fix(benchmark): align MLA metadata with backend contracts by @lunarz-dev in https://github.com/flashinfer-ai/flashinfer/pull/4835
- Add PR labeler rules for op: linear attention and op: misc by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4852
- [bug fix] Issue 3700 sparse mla sm121 hang by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4732
- feat(jit): log cold nvcc module builds by @mgoin in https://github.com/flashinfer-ai/flashinfer/pull/4844
- docs: fix 0.6.19 document check failures by @kangbintNV in https://github.com/flashinfer-ai/flashinfer/pull/4871
- Rubin open issue fixes by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4787
- fix(attention): count NVFP4 scale-factor staging in the batch smem budget by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/4767
- fix(attention): scope the NVFP4 split-KV workaround to the arch it was found on by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/4747
- fix: honor sparse MLA KV page strides by @bvolpato in https://github.com/flashinfer-ai/flashinfer/pull/4362
- feat(gemm): enable tinygemm2_sm100 on SM107 (Rubin) by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4849
- test(moe): generate activation matrix and add backend contract fuzzers by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4805
- refactor(moe): bind TRTLLM launch state to each call by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4797
- fix(cudnn): return base-2 LSE from prefill (+ paged head_dim_qk != head_dim_vo O offset) by @egilliam-nv in https://github.com/flashinfer-ai/flashinfer/pull/4663
- feat: Add cute-dsl-prims backend for SM120 FP8 GQA prefill by @Tom-Zheng in https://github.com/flashinfer-ai/flashinfer/pull/4714
- Enable top_k_varlen on Rubin (SM107) and add the DKG filtered-radix backend (
radix_filter) by @dhiraj113 in https://github.com/flashinfer-ai/flashinfer/pull/4621 - chore: update TRTLLM-Gen GEMM cubins by @bobboli in https://github.com/flashinfer-ai/flashinfer/pull/4840
- feat(moe): add CuTe DSL SM90 BF16 MoE backend by @Aneureka in https://github.com/flashinfer-ai/flashinfer/pull/4878
- fix(gemm,moe): restore bmm_fp8 auto fallback and drop an over-strict SM107 tactic guard by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4853
- docs(moe): regenerate the Unified MoE activation matrix (unblocks CI on main) by @Anerudhan in https://github.com/flashinfer-ai/flashinfer/pull/4903
- feat(moe): enable tile_size=256 for Rubin MoE autotuning by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4851
- perf(sparse mla): Update sparse MLA half-Q-tile heuristic to apply to short queries only by @emascarenhas-nv in https://github.com/flashinfer-ai/flashinfer/pull/4752
- fix(gdn): stream-scope WY staging buffers and prefill launch workspace by @elwhyjay in https://github.com/flashinfer-ai/flashinfer/pull/4476
- fix(moe): disable PDL in the trtllm-gen fused-MoE path on SM107 (Rubin) by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4806
- feat(moe_ep): SM120 MXFP8 swap-AB CuTeDSL MegaMoE kernel by @mhoqueanik in https://github.com/flashinfer-ai/flashinfer/pull/4387
- feat(moe_ep): make the nccl_ep split path CUDA-graph capturable by @Anerudhan in https://github.com/flashinfer-ai/flashinfer/pull/4795
- feat(gemm): add router GEMM ops for Kimi-K2 / Kimi-K3 and bf16 output by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4630
- feat(prims_ts): accept SM107 (Rubin) in the attention-ts device gate by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4755
- feat(moe): expand cuTile fused MoE activation support by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4888
- fix(gemm): remove SM120 MXFP8 tile configs that exceed the SM12x shared-memory limit by @yichengj0 in https://github.com/flashinfer-ai/flashinfer/pull/4013
- Add CuteDSL Low Latency GEMM tactic to mm_fp4, mm_fp8, mm_mxfp8, tgv_gemm_sm100 backends (SM100/SM103) by @PetersonGuo in https://github.com/flashinfer-ai/flashinfer/pull/4685
- feat(cake_warp_decode): add SM103 NVFP4 warp-decode backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4855
- feat(cake_comm): extend Blackwell all-gather matmul to TP8 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4822
- Summary improvement by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/4895
- fix(kda): accept SM107 (Rubin) in the FlashKDA family guards by @Vinnie6167 in https://github.com/flashinfer-ai/flashinfer/pull/4710
- ci: enable patched sccache for CUDA 13.4 by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4808
- Modify CODEOWNERS - Anik by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4937
- feat(kda): output-only KDA decode + drop-in for vLLM Kimi K3 recurren… by @ameynaik-hub in https://github.com/flashinfer-ai/flashinfer/pull/4709
- Refactor Sparse MLA SM120 by @lucifer1004 in https://github.com/flashinfer-ai/flashinfer/pull/4802
- Add ReplaySSM autotuning by @askliar in https://github.com/flashinfer-ai/flashinfer/pull/4545
- feat(attention): cuTile paged/ragged prefill + paged/MLA decode by @yifeis-nv in https://github.com/flashinfer-ai/flashinfer/pull/4018
- docs: teach agents the PR guidelines in CLAUDE.md by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4948
- [feat] Add warp level split k bf16 gemm by @jiahanc in https://github.com/flashinfer-ai/flashinfer/pull/4908
- feat: add experimental API/backend policy,
@flashinfer_experimental_api, andflashinfer.experimentalnamespace by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/4880 - fix(ci): publish the test scope with GITHUB_TOKEN, not the bot PAT by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4962
- test: skip MonoMoE on GPUs with fewer than 128 SMs by @cindyzxq in https://github.com/flashinfer-ai/flashinfer/pull/4954
- feat(topk): self-sampling GVR V2 backend (SM100/103/107), oracle-tracking auto, and the V1 threshold repair by @dhiraj113 in https://github.com/flashinfer-ai/flashinfer/pull/4811
- ci: provision cuTile compiler in CUDA 13 images by @saltyminty in https://github.com/flashinfer-ai/flashinfer/pull/4939
- Add option to skip checking active rows in TRTLLM ragged prefill by @wzhao18 in https://github.com/flashinfer-ai/flashinfer/pull/4931
- fix(ci): anchor bot command matching to line start, outside code fences by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4956
- feat(autotuner): Autotuner v2 (autotune_v2) — managed persistence, deployment-matched measurement, runner contract by @YangXu1990uiuc in https://github.com/flashinfer-ai/flashinfer/pull/3861
- feat(cake_concat_mla): add SM100/SM103 concat MLA K backend by @Kathryn-cat in https://github.com/flashinfer-ai/flashinfer/pull/4860
- feat(cake_kda): add fp32 state & long context kda prefill kernels by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4845
- feat(cake_sage): add SM120 block-sparse attention backend by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4951
- feat(prims-ts): support proxy-compensated block-sparse attention by @heyuhhh in https://github.com/flashinfer-ai/flashinfer/pull/4872
- feat(prims-ts): support decode GQA ratios up to 128 by @PerkzZheng in https://github.com/flashinfer-ai/flashinfer/pull/4915
- feat(mla): expose DSv4 TRTLLM-GEN RopeQuant by @PerkzZheng in https://github.com/flashinfer-ai/flashinfer/pull/4918
- fix(moe): address review findings for PR [#4610] by @cindyzxq in https://github.com/flashinfer-ai/flashinfer/pull/4914
- perf(attention): repack NVFP4 KV tiles to 16-bit on targets without a native E2M1 convert by @lesj0610 in https://github.com/flashinfer-ai/flashinfer/pull/4769
- feat(sm120): add NVFP4 sparse MLA support for DeepSeek V4 Flash by @tiffany940107 in https://github.com/flashinfer-ai/flashinfer/pull/4955
- Refactor SM120 grouped GEMM as SM12x GEMM by @CarstyYou in https://github.com/flashinfer-ai/flashinfer/pull/4838
- feat: add NVFP4 quantize-append path for the MLA paged KV cache by @elwhyjay in https://github.com/flashinfer-ai/flashinfer/pull/4676
- docs: resolve documentation check failures by @kangbintNV in https://github.com/flashinfer-ai/flashinfer/pull/4927
- perf(moe): phase NVFP4 dispatch payload copies at EP4/EP8/EP16 by @samnordmann in https://github.com/flashinfer-ai/flashinfer/pull/4067
- fix(docs): deploy stable documentation from release tags by @cindyzxq in https://github.com/flashinfer-ai/flashinfer/pull/4945
- fix(kda): compile direct serving modules as C++20 by @migarci2 in https://github.com/flashinfer-ai/flashinfer/pull/5017
- feat(cake_mega_moe): add optimized MXFP8 MegaMoE EP16 backend by @hzfan in https://github.com/flashinfer-ai/flashinfer/pull/4970
- feat: Add valid_hidden_size and valid_intermediate_size params t… by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/2482
- Nccl extensions dependency update by @Anerudhan in https://github.com/flashinfer-ai/flashinfer/pull/5016
- fix(trace): model the SM100/103 cute-dsl mm_bf16_fp4 prepared layout by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/4620
- fix(mla): scope NoPE sparse_mla_top_k_lens requirement to non-sparse backends by @lucifer1004 in https://github.com/flashinfer-ai/flashinfer/pull/4947
- fix(moe): unify the NVFP4 profiler workspace predicate and fix gated fc1 SF sizing by @yichengj0 in https://github.com/flashinfer-ai/flashinfer/pull/4010
- feat(gdn): adopt the CuTe-DSL disk cache for TVM-FFI GDN kernels by @elwhyjay in https://github.com/flashinfer-ai/flashinfer/pull/4912
- fix(gdn): support WY ucache kernels on SM121 by @hebo1221 in https://github.com/flashinfer-ai/flashinfer/pull/4528
- feat(cake_gdn): add gdn prefill decode mtp Cake backend for SM100 and SM103 by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/4581
- perf(prims-ts): Optimize&refine PrimsTS block sparse attention by @heyuhhh in https://github.com/flashinfer-ai/flashinfer/pull/5002
- feat(attention): opt-in LDTM.STAT for PrimTS context row_max by @kris1025 in https://github.com/flashinfer-ai/flashinfer/pull/4839
- chore: remove unused benchmark test by @yufeiwu-nv in https://github.com/flashinfer-ai/flashinfer/pull/5046
- feat(moe): wire unified UnpackedPrecomputed for TRTLLM BF16 and block-FP8 by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4909
- feat(kda): support graph-safe CuTe DSL prefix checkpoints by @Observer007 in https://github.com/flashinfer-ai/flashinfer/pull/5021
- feat(benchmarks): measure TRTLLM ragged row-check overhead by @yufeiwu-nv in https://github.com/flashinfer-ai/flashinfer/pull/4998
- test(kda): restore the frozen module-ident consistency check by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/5035
- feat(cake_gqa): Add an experimental SM110 GQA decode kernel by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/5052
- feat(moe): split QuantConfig into weight/activation/output format axes by @feih-nv in https://github.com/flashinfer-ai/flashinfer/pull/4952
- fix(decode): reserve max(K+V, st.o) smem for FA2 FP8+GQA decode by @ir1ka in https://github.com/flashinfer-ai/flashinfer/pull/5038
- fix: apply calibration scales to ragged FP8 KV prefill by @yuchenwang3 in https://github.com/flashinfer-ai/flashinfer/pull/4984
- perf(attention): Refresh DSL_FMHA cubins by @xrq-phys in https://github.com/flashinfer-ai/flashinfer/pull/4997
- fix(bench): quote benchmark CSV fields by @lunarz-dev in https://github.com/flashinfer-ai/flashinfer/pull/5045
- fix(attention): enable SM121 FMHA v2 prefill by @aeichler-ac in https://github.com/flashinfer-ai/flashinfer/pull/4661
- fix(mla): prevent intermittent hang in SM120 sparse-MLA swapAB prefill on DGX Spark by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/5048
- refactor(attention): unify PrimTS plan/run and page-table contracts by @yuxianq in https://github.com/flashinfer-ai/flashinfer/pull/4829
- Replayssm prefix materialize by @hnover-nv in https://github.com/flashinfer-ai/flashinfer/pull/4815
- fix(bench): use float32 cos_sin_cache in apply_rope_with_cos_sin_cache by @aeichler-ac in https://github.com/flashinfer-ai/flashinfer/pull/5066
- feat(cake_kernel): add MiniMax-H3 MXFP8 pre-attention for SM100a/SM103a by @yyihuang in https://github.com/flashinfer-ai/flashinfer/pull/5060
- fix(jit): avoid invalidating bgmv_moe cache on source staging by @200lz in https://github.com/flashinfer-ai/flashinfer/pull/4798
- ci: log installed Python packages before tests by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/5062
- fix(jit): preserve source mtimes when staging rmsnorm_silu and monomoe sources by @bkryu in https://github.com/flashinfer-ai/flashinfer/pull/5063
- fix(kda): make recurrent_kda backend="auto" decode fall back to CuTe DSL by @kahyunnam in https://github.com/flashinfer-ai/flashinfer/pull/5037
- feat: Test runner summary prints cudnn backend version by @righthandabacus in https://github.com/flashinfer-ai/flashinfer/pull/5068
- feat(jit-cache): split cache into architecture provider wheels by @dierksen in https://github.com/flashinfer-ai/flashinfer/pull/4514
- bump version to 0.7.0 by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/4938
- fix(release): drop the stale diffusion_ops.minimax_h3 trace-registry entry by @aleozlx in https://github.com/flashinfer-ai/flashinfer/pull/5216
New Contributors
- @shoutoutuoadi325 made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4401
- @JimpleMa made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4633
- @NaderAlAwar made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4442
- @mingxu1067 made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4599
- @PetersonGuo made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4793
- @bvolpato made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4362
- @emascarenhas-nv made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4752
- @Kathryn-cat made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4860
- @migarci2 made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/5017
- @kris1025 made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4839
- @yufeiwu-nv made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/5046
- @yuchenwang3 made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4984
- @aeichler-ac made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4661
- @hnover-nv made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4815
- @200lz made their first contribution in https://github.com/flashinfer-ai/flashinfer/pull/4798
Full Changelog: https://github.com/flashinfer-ai/flashinfer/compare/v0.6.18rc10...v0.7.0
