DeepSpeed Files

What's Changed

• Update version after latest release (v0.18.9) by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7936
• Refactor consolidate transpose by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7934
• Fix/fix autotp universal checkpoint ci by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7937
• Fix process hang in process-group shutdown by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7941
• Zero3 defragment utility by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7940
• [SP] add SP deny list instead of allow by @kashif in https://github.com/deepspeedai/DeepSpeed/pull/7887
• fix(zero): detach flat buffer to prevent autograd inplace error on CP… by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7948
• Fix FPQuantizer build by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7963
• Fix zero 1 and 2 CPU-offloaded gradient norm by @alek6kun in https://github.com/deepspeedai/DeepSpeed/pull/7967
• Fix overlap-comm buffer lifetimes by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7965
• Fix DeepCompile+Z3 on PyTorch v2.9/2.10 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7951
• Fix WarmupCosineLR multi-group initialization by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7969
• Enable PyTorch version selection for full test by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7968
• fix(fp_quantizer): fix UB and negative shift warnings in fp_quantize_impl.cu by @Cursx in https://github.com/deepspeedai/DeepSpeed/pull/7973
• fix(op_builder): avoid duplicate/wrong -gencode flags by @Cursx in https://github.com/deepspeedai/DeepSpeed/pull/7974
• Rename dequantization template parameters by @Flamefire in https://github.com/deepspeedai/DeepSpeed/pull/7976
• Avoid CUDA reinit error in CI tests by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7977
• Fix ZeRO-1/2 CPU-offloaded gradient loss with multiple backward() per step by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/7981
• deepcompile: Fix backward graph recompilation due to unbalanced forward/backward visits by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7980
• Fix Adam subgroup inconsistency by @st-bang97 in https://github.com/deepspeedai/DeepSpeed/pull/7982
• Dynamic offload compatible with static optimizer offload by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7979
• Fix modal ci timeout by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7989
• Fix BF16_Optimizer last-microbatch grad leak under ZeRO-1 by @maxyu1115 in https://github.com/deepspeedai/DeepSpeed/pull/7985
• fix: topkgating major bug by @excepshenal in https://github.com/deepspeedai/DeepSpeed/pull/7986
• Add DeepSpeed NVTX domain support by @heurry in https://github.com/deepspeedai/DeepSpeed/pull/7988
• Add Gram Newton-Schulz orthogonalization for Muon optimizer by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7953
• [AutoSP] (Sequence Parallelism) support for Multimodal Models (ViT + LLM) by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/7984
• Update version.txt before 0.19.0 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7995

New Contributors

• @alek6kun made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7967
• @Cursx made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7973
• @roycho96 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7981
• @st-bang97 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7982
• @maxyu1115 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7985
• @excepshenal made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7986
• @heurry made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7988

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.18.9...v0.19.0