Liger Kernel Files

Overview

Liger-Kernel v0.8.2 delivers performance, correctness, and numerical-precision improvements across cross entropy, MoE, RMSNorm, JSD, and other operators. The release also improves Megatron integration, testing, and overall kernel robustness.

This release further expands experimental backends and features, including CuTe DSL, cuTile, and LigerCute, with new kernels and optimizations targeting Hopper and Blackwell GPUs.

Summary

• Added fused linear scaled cross entropy, providing an efficient frontend compatible with verl-style fused log-probability computation for RL training. See [verl's fused linear cross-entropy implementation](https://github.com/volcengine/verl/blob/main/verl/utils/kernel/linear_cross_entropy.py?utm_source=chatgpt.com).
• Improved performance and numerical accuracy of cross-entropy kernels.
• Fixed correctness and robustness issues across cross entropy, RMSNorm, JSD, and MoE.
• Added Megatron SwiGLU integration and improved NPU support.
• Expanded experimental CuTe DSL, cuTile, and LigerCute support, including additional Hopper and Blackwell kernels.

What's Changed

• Pin ruff version to <0.16.0 to avoid checkstyle breakage by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/1318
• Guard the FLCE addmm fast path on compute capability 8.0 or higher by @albertvillanova in https://github.com/linkedin/Liger-Kernel/pull/1316
• feat: Add accum dtype support to fused linear JSD by @KyleMylonakisProtopia in https://github.com/linkedin/Liger-Kernel/pull/1322
• fix(test): prevent in-place RMSNorm backward from clobbering shared grad buffers by @yueyiming2009 in https://github.com/linkedin/Liger-Kernel/pull/1305
• perf(ce): single-shot dx_y gradient correction by @justinhh4 in https://github.com/linkedin/Liger-Kernel/pull/1268
• Fix wrong grad_weight/grad_bias for fused linear cross entropy with reduction='none' by @kashif in https://github.com/linkedin/Liger-Kernel/pull/1298
• Guard Triton launches by tensor device by @kashif in https://github.com/linkedin/Liger-Kernel/pull/1304
• fix(ce): restore true-class gradient precision in low-precision buffers by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/1329
• [CuteDSL] Add benchmarking, CE memory optimizations, and match Triton num_warps by @justinhh4 in https://github.com/linkedin/Liger-Kernel/pull/1290
• [Tests] Move cutedsl tests to test/cutedsl/ and add make test-cutedsl target by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/1337
• Add Hopper CuTe DSL fused scaled cross entropy by @kolehma8 in https://github.com/linkedin/Liger-Kernel/pull/1334
• Add SM100 CuTe DSL fused linear cross entropy by @justinhh4 in https://github.com/linkedin/Liger-Kernel/pull/1339
• perf(cutedsl): optimize RMSNorm CuTe DSL kernel (fused backward + host launch + width-aware forward warps) by @Charlesrizzly in https://github.com/linkedin/Liger-Kernel/pull/1349
• fix(triton): pin fp64 scalar kernel params to fp32 in swiglu and rms_norm by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/1350
• Add LigerCute: fused expert-parallel MoE kernels for Hopper and Blackwell (CUTLASS + NVSHMEM) by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/1355
• Add cutedsl implementation of Liger Rope Kernel by @Celaena24 in https://github.com/linkedin/Liger-Kernel/pull/1282
• Add fused linear scaled cross entropy frontend by @kolehma8 in https://github.com/linkedin/Liger-Kernel/pull/1354
• Add CuTe DSL implementation of the swiglu liger kernel by @Celaena24 in https://github.com/linkedin/Liger-Kernel/pull/1277
• [cuTile] Add fused scaled cross entropy by @yueyiming2009 in https://github.com/linkedin/Liger-Kernel/pull/1359
• [Megatron] Add SwiGLU integration by @buffett0323 in https://github.com/linkedin/Liger-Kernel/pull/1326
• [Liger Cute] Fix fused MoE combine for partial token tiles by @supercharleszhu in https://github.com/linkedin/Liger-Kernel/pull/1361
• perf(moe): add Blackwell 2SM MLP3 and MLP4 by @autinn in https://github.com/linkedin/Liger-Kernel/pull/1365
• perf(cutedsl): optimize Hopper FLSCE forward by @kolehma8 in https://github.com/linkedin/Liger-Kernel/pull/1370
• [Hopper] Add CuTe DSL and cuTile fused linear cross entropy by @Celaena24 in https://github.com/linkedin/Liger-Kernel/pull/1356
• Fix single-token fused MoE auto dispatch by @supercharleszhu in https://github.com/linkedin/Liger-Kernel/pull/1368
• [NPU] Fix: recompute true-class CE gradient in fp32 on no-weight backward path by @zheliuyu in https://github.com/linkedin/Liger-Kernel/pull/1341
• Pin fp64 scalar kernel params to fp32 in cross entropy by @buffett0323 in https://github.com/linkedin/Liger-Kernel/pull/1362
• Pin fp64 scalar kernel params via constexpr in grpo_loss by @buffett0323 in https://github.com/linkedin/Liger-Kernel/pull/1366
• Pin fp64 scalar kernel params to fp32 in remaining norm ops by @buffett0323 in https://github.com/linkedin/Liger-Kernel/pull/1358
• Pin fp64 scalar kernel params to fp32 in modulated_rms_norm and attn_res by @buffett0323 in https://github.com/linkedin/Liger-Kernel/pull/1364
• Support strided Liger MoE input weights by @supercharleszhu in https://github.com/linkedin/Liger-Kernel/pull/1377
• Tune SM100 GEMM scheduling for dW shapes by @justinhh4 in https://github.com/linkedin/Liger-Kernel/pull/1380
• Optimize and harden fused MoE kernels by @kolehma8 in https://github.com/linkedin/Liger-Kernel/pull/1381
• Update CuTe DSL backend docs: op list and DSL version by @PKUWZP in https://github.com/linkedin/Liger-Kernel/pull/1386
• perf(cutedsl): add Blackwell fast path to RMSNorm by @PKUWZP in https://github.com/linkedin/Liger-Kernel/pull/1388
• [NPU] align fused linear JSD forward with accum_dtype API by @zheliuyu in https://github.com/linkedin/Liger-Kernel/pull/1346
• Update pyproject.toml for v0.8.2 release by @kolehma8 in https://github.com/linkedin/Liger-Kernel/pull/1394

New Contributors

• @KyleMylonakisProtopia made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/1322
• @buffett0323 made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/1326
• @supercharleszhu made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/1361
• @autinn made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/1365

Full Changelog: https://github.com/linkedin/Liger-Kernel/compare/v0.8.1...v0.8.2