Download Latest Version
v0.22.0 source code.tar.gz (28.4 MB)
Get an email when there's a new version of LLM Foundry
| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| README.md | 2025-05-31 | 3.3 kB | |
| v0.21.0 source code.tar.gz | 2025-05-31 | 28.4 MB | |
| v0.21.0 source code.zip | 2025-05-31 | 28.6 MB | |
| Totals: 3 Items | 57.1 MB | 0 | |
TLDR
- Torch version has been bumped to 2.7.0
- Support FSDP2 via a ENV VAR: FSDP_VERSION=2. Currently it only supports pretraining (with meta init). No yaml change is needed to enable FSDP2, the attrs that only apply to FSDP(1) will be ignored and raised as warnings. See composer release for more details
What's Changed
- Adding support for nope positional encoding in block overrides. by @ShashankMosaicML in https://github.com/mosaicml/llm-foundry/pull/1794
- Bump foundry version to 0.21.0.dev0 by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1812
- Adding temperature tuning in attention by @ShashankMosaicML in https://github.com/mosaicml/llm-foundry/pull/1793
- Update foundry version in MCLI yamls by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1813
- Upgrade yapf version by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1814
- Allow subselecting the appropriate config for llama4 by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1815
- Change RMSNorm to use PyTorch native implementation by @josejg in https://github.com/mosaicml/llm-foundry/pull/1809
- Update datasets requirement from <3.6,>=3.3.2 to >=3.3.2,<3.7 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1817
- Bump onnxruntime from 1.19.2 to 1.22.0 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1819
- Update huggingface-hub[hf_xet] requirement from <0.31,>=0.30.0 to >=0.30.0,<0.32 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1818
- Deprecate inference API wrappers by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1821
- Fix Dtensor initialization by @bowenyang008 in https://github.com/mosaicml/llm-foundry/pull/1820
- Update accelerate requirement from <1.7,>=0.25 to >=0.25,<1.8 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1824
- Bump onnx from 1.17.0 to 1.18.0 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1823
- Bump docformatter for python3.12 and change blank_line_before_module_docstring = false by @sashaDoubov in https://github.com/mosaicml/llm-foundry/pull/1825
- Delete useless print("here") by @tsebaka in https://github.com/mosaicml/llm-foundry/pull/1826
- Update ci-testing version to latest by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1827
- Bump coverage[toml] from 7.8.0 to 7.8.2 by @dependabot in https://github.com/mosaicml/llm-foundry/pull/1830
- Configurable shard size by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1833
- Bump Composer 0.31.0 by @bowenyang008 in https://github.com/mosaicml/llm-foundry/pull/1835
- Fix monolithic checkpointing against composer main by @dakinggg in https://github.com/mosaicml/llm-foundry/pull/1836
- Bump torch version to 2.7 by @bowenyang008 in https://github.com/mosaicml/llm-foundry/pull/1832
- bump huggingface-hub upper bound to 0.33 by @bowenyang008 in https://github.com/mosaicml/llm-foundry/pull/1838
New Contributors
- @bowenyang008 made their first contribution in https://github.com/mosaicml/llm-foundry/pull/1820
- @tsebaka made their first contribution in https://github.com/mosaicml/llm-foundry/pull/1826
Full Changelog: https://github.com/mosaicml/llm-foundry/compare/v0.20.0...v0.21.0