Liger Kernel Files

Highlights in this release:

New model architecture supports: SmolVLM2, GLM4.5V, InternVL3, Falcon-H1, Qwen-Next

New algorithm: GSPO

What's Changed

• [cross-entropy-loss] Added support for DFT flag by @kashif in https://github.com/linkedin/Liger-Kernel/pull/860
• fix(test): update assertions in GLM4 instance patching tests by @vvvdwbvvv in https://github.com/linkedin/Liger-Kernel/pull/859
• Fix nan loss error for LigerFusedLinearJSDLoss by @ParagEkbote in https://github.com/linkedin/Liger-Kernel/pull/862
• [Cross-entropy] get valid predicted probabilities by @kashif in https://github.com/linkedin/Liger-Kernel/pull/864
• Enhance Docs by @ParagEkbote in https://github.com/linkedin/Liger-Kernel/pull/867
• Add Classifiers for Liger-Kernel by @ParagEkbote in https://github.com/linkedin/Liger-Kernel/pull/869
• docs(mta): suppress invalid sequence syntax warning by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/870
• Add GSPO by @BjarniHaukur in https://github.com/linkedin/Liger-Kernel/pull/845
• Add GLM4.5V support by @vvvdwbvvv in https://github.com/linkedin/Liger-Kernel/pull/863
• A Fix for Issue [#872] by @yshenaw in https://github.com/linkedin/Liger-Kernel/pull/879
• Add pytest coverage for liger-kernel by @ParagEkbote in https://github.com/linkedin/Liger-Kernel/pull/876
• Replace all torch_dtype with dtype by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/881
• Update Dev Dependencies by @ParagEkbote in https://github.com/linkedin/Liger-Kernel/pull/886
• Fixed AMD CI issue [#793] by @DevManpreet5 in https://github.com/linkedin/Liger-Kernel/pull/887
• fix(layernorm): remove n_cols upcasting for for torch.compile by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/884
• Fix tests and CI by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/882
• Remove daily test cron job by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/890
• [UT] [XPU] Modify the test cases of XPU for triton3.5 by @YangKai0616 in https://github.com/linkedin/Liger-Kernel/pull/889
• Add InternVL3 support by @MilkClouds in https://github.com/linkedin/Liger-Kernel/pull/878
• fix(flce): add shift_labels as eval mode loss condition by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/888
• Add support of Falcon-H1 models for liger kernels by @puneeshkhanna in https://github.com/linkedin/Liger-Kernel/pull/900
• Don't deploy mkdocs to fix benchmarking by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/904
• Disable mllama multimodal test in transformers<4.51.0 by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/899
• Add flce forward for FalconH1ForCausalLM and missing tests by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/903
• feat(ce,flce): decouple gradients computation for no_grad mode by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/894
• fix(llama4): Get correct swiglu patch target for llama4 moe layer by @alenawang in https://github.com/linkedin/Liger-Kernel/pull/907
• Add PolyNorm operator by @0xtoward in https://github.com/linkedin/Liger-Kernel/pull/901
• Copy and paste benchmarks before and after gh-pages deployment by @vaibhavjindal in https://github.com/linkedin/Liger-Kernel/pull/909
• Filter out redundant ops/allocations in no_grad mode by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/906
• Add support for Qwen3Next model with Liger kernels by @vvvdwbvvv in https://github.com/linkedin/Liger-Kernel/pull/912
• refactor(convergence-test): remove unnecessary print by @Tcc0403 in https://github.com/linkedin/Liger-Kernel/pull/913
• Enabled the tests glm4v/glm4v_moe for XPU and Fixed the monkey patch error by @YangKai0616 in https://github.com/linkedin/Liger-Kernel/pull/914
• [Test][XPU] Added gpu cache cleaning for XPU devices by @Egor-Krivov in https://github.com/linkedin/Liger-Kernel/pull/917
• Add SmolVLM2 support by @MilkClouds in https://github.com/linkedin/Liger-Kernel/pull/919

New Contributors

• @BjarniHaukur made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/845
• @yshenaw made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/879
• @DevManpreet5 made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/887
• @MilkClouds made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/878
• @puneeshkhanna made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/900
• @alenawang made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/907
• @0xtoward made their first contribution in https://github.com/linkedin/Liger-Kernel/pull/901

Full Changelog: https://github.com/linkedin/Liger-Kernel/compare/v0.6.2...v0.6.3