Download Latest Version v0.6.1 source code.tar.gz (3.7 MB)
Email in envelope

Get an email when there's a new version of Liger Kernel

Home / v0.6.0
Name Modified Size InfoDownloads / Week
Parent folder
README.md 2025-07-09 2.5 kB
0
v0.6.0_ New Attention Operators, Cosine Similarity Loss, Llama 4, and VLM Patching Updates source code.tar.gz 2025-07-09 3.7 MB
0
v0.6.0_ New Attention Operators, Cosine Similarity Loss, Llama 4, and VLM Patching Updates source code.zip 2025-07-09 3.9 MB
0
Totals: 3 Items   7.6 MB 0

Highlights

This release introduces significant improvements to Liger-Kernel, including new operators, support for Llama 4 models, more robust benchmarking automation, and key fixes for patching of vision-language models (VLMs) due to recent transformers refactoring.

Key Changes

New Features & Improvements

  • Multi-Token Attention by @AndreSlavescu (#689)
  • Fused Neighborhood Attention by @AndreSlavescu (#732)
  • Cosine Similarity Loss for Distillation by @Dexterai (#780)
  • Support for Llama 4 by @Manan17 (#740)
  • Option to choose fused LCE/CE loss by @connermanuel (#704)
  • Add block_rms_norm for QK norm by @mdy666 (#731)

Bug Fixes

  • Vision-language model patching in recent transformers versions (>=4.52.0):
  • Qwen2vl, Qwen2_5_vl by @Tcc0403 (#728, [#738])
  • Llava by @Tcc0403, @Manan17 (#714, [#743], [#751])
  • Gemma3 by @shimizust, @Tcc0403 (#735, [#787], [#790]);
  • RMS Norm patching by @vaibhavjindal, @BenasdTW (#741, [#765])
  • Hugging Face forward kwargs fix by @llllvvuu (#708)
  • Fix import tanh by @jue-jue-zi (#762)
  • Apply monkey patch to instances by @YangKai0616 (#772)

Documentation & CI Fixes

  • Deploy MkDocs to GitHub Pages by @ParagEkbote (#724)
  • Robust doc updates by @ParagEkbote (#726, [#727])
  • .idea ignored by @Tcc0403 (#784)
  • ReadMe, MTA + softmax docs by @AndreSlavescu (#730)
  • Relax DyT tol, XPU skip MTA by @Tcc0403 (#778)
  • Paligemma test fixes by @vvvdwbvvv (#785)
  • Style & test fixes by @Tcc0403, @vaibhavjindal (#736, [#794])
  • Add torchvision for multimodal test by @Tcc0403 (#755)

Benchmarking & Automation

  • Automated benchmarking and visualization UI in GitHub pages by @Manan17 (#744, [#747], [#749], [#752], [#753], [#756], [#759], [#760], [#770], [#779])

New Contributors

Full Changelog: https://github.com/linkedin/Liger-Kernel/compare/v0.5.10...v0.6.0

Source: README.md, updated 2025-07-09