BioNeMo Files

New Features

• Adds support for AMPLIFY doi:10.1101/2024.09.23.614603 pre-training and inference, offering a 70% speedup over the xformers-based attention backend with similar final perplexity values at 1M pre-training steps. (4.23 for 120M, 3.05 for 350M). The model is fully compatible with existing weights on HuggingFace.
• Adds alpha support for LoRA fine-tuning to for ESM2 models. Inference and fine-tuning are enabled along with resumption from a checkpoint.

Updates & Improvements

• Blackwell support, tested on B200 systems.
• Fixed Grace CPU support, released ARM compatible container.

What's Changed

• hotfix: docker build in CI by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/756
• updated version for 2.5 release by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/755
• update evo2 partial conv max steps by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/736
• bump ruff to 0.9.10 and reformat files as necessary by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/751
• Updated file paths for images in SCDL README by @polinabinder1 in https://github.com/NVIDIA/bionemo-framework/pull/758
• hotfix CI: failing test test_train_evo2_stops by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/761
• adding v2.5 release notes by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/764
• switch to GHA runners by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/734
• Jwilber/update evo2 readme and assets by @jwilber in https://github.com/NVIDIA/bionemo-framework/pull/759
• Aligning directories with tensorboard logs for ESM2 and Evo2 by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/740
• Remove Evo2 PR announcement now that everything is merged by @jstjohn in https://github.com/NVIDIA/bionemo-framework/pull/772
• switch cache-from flags by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/773
• Add AMPLIFY model and huggingface conversion scripts by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/640
• parallelize test stages in GitHub CI by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/768
• D3pm blackwell testing stability fix by @nvdreidenbach in https://github.com/NVIDIA/bionemo-framework/pull/743
• fix: attempts to update geneformer notebooks by @jomitchellnv in https://github.com/NVIDIA/bionemo-framework/pull/745
• [hotfix] setting as "ignore" the failing notebook geneformer_cellxgene_tutorial.ipynb by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/779
• Remove xformers install by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/781
• Remove outdated evo2 tutorial that is now in the submodule by @jstjohn in https://github.com/NVIDIA/bionemo-framework/pull/783
• Blackwell compatibility changes by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/707
• Add AMPLIFY inference by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/775
• Pin griffe to 1.6.2 by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/789
• hotfix: evo2 divergence - downgrade TE to v1.13 by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/791
• Updates tflops chart for Geneformer. by @jomitchellnv in https://github.com/NVIDIA/bionemo-framework/pull/785
• change checkpoint name pattern by @farhadrgh in https://github.com/NVIDIA/bionemo-framework/pull/786
• Revert commit [67a869] (TE_VERSION=v1.13 fix) by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/795
• [cye/subpack-gpu-testing] Add GPU runner to testing job. by @cspades in https://github.com/NVIDIA/bionemo-framework/pull/776
• Dockerfile improvements for ARM by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/777
• Remove llama-index from container to fix CVEs by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/800
• Bump 3rdparty/NeMo from cc8ff45 to 384ff02 by @dependabot in https://github.com/NVIDIA/bionemo-framework/pull/792
• Add local clone script by @nvdreidenbach in https://github.com/NVIDIA/bionemo-framework/pull/787
• Fix ARM docker build by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/801
• [cye/ml-subpackage-ci] Onboard bionemo-llm and bionemo-noodles to the sub-package CI. by @cspades in https://github.com/NVIDIA/bionemo-framework/pull/809
• Update README.md link by @nvdreidenbach in https://github.com/NVIDIA/bionemo-framework/pull/812
• Have dependabot update our docker base image by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/813
• Add .codecov.yml status checks by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/618
• Add AMPLIFY model documentation, minor type fixes by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/788
• Remove import guard in bionemo-llm by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/804
• Bump rust from 1.82.0 to 1.86.0 by @dependabot in https://github.com/NVIDIA/bionemo-framework/pull/819
• Bump crossbeam-channel from 0.5.13 to 0.5.15 in /sub-packages/bionemo-noodles by @dependabot in https://github.com/NVIDIA/bionemo-framework/pull/818
• Pbinder/geneformer partial conv by @polinabinder1 in https://github.com/NVIDIA/bionemo-framework/pull/802
• [cye/rapids-sc-install] Add rapids_singlecell import to BioNeMo FW container image. by @cspades in https://github.com/NVIDIA/bionemo-framework/pull/816
• Biopharma mailing list docs addition. by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/822
• unify the implementation of early training termination across BioNeMo subpackages and update benchmarks by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/803
• Fix bitsandbytes issue on ARM by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/824
• Fixes for AMPLIFY QA scripts by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/825
• updated configs for benchmarks by @dorotat-nv in https://github.com/NVIDIA/bionemo-framework/pull/833
• Remove temporary pins in docs build by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/828
• Adding baseline metrics for benchmarking ESM2 model by @ShevaNguyen in https://github.com/NVIDIA/bionemo-framework/pull/831
• Updates docs for geneformer training, inference, and cellxclassification by @jomitchellnv in https://github.com/NVIDIA/bionemo-framework/pull/823
• Add pre commit to verify test status by @pstjohn in https://github.com/NVIDIA/bionemo-framework/pull/841
• fix geneformer image paths by @jomitchellnv in https://github.com/NVIDIA/bionemo-framework/pull/839
• fix geneformer image links by @jomitchellnv in https://github.com/NVIDIA/bionemo-framework/pull/844
• ESM2 PEFT by @polinabinder1 in https://github.com/NVIDIA/bionemo-framework/pull/766
• Pbinder/esm2 document by @polinabinder1 in https://github.com/NVIDIA/bionemo-framework/pull/846
• h11 CRIT vuln fix by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/847
• Docs fix by @trvachov in https://github.com/NVIDIA/bionemo-framework/pull/826

New Contributors

• @ShevaNguyen made their first contribution in https://github.com/NVIDIA/bionemo-framework/pull/831

Full Changelog: https://github.com/NVIDIA/bionemo-framework/compare/v2.5...v2.6