DeepSeed Files

What's Changed

• Update version post-v0.19.0 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7996
• Add office hours times/link on the README by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/8004
• Update topkgating probability-mask test expectation by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8007
• Optimize singleton MoE collectives by @Tianyi-Franklin-Wang in https://github.com/deepspeedai/DeepSpeed/pull/7997
• zero3: SDMA allgather via mori (sdma_allgather) by @inkcherry in https://github.com/deepspeedai/DeepSpeed/pull/7999
• fix(io): close aio_fd in FastFileWriter._fini to prevent fd leak by @jg-heo in https://github.com/deepspeedai/DeepSpeed/pull/8005
• Auto-detect CUTLASS for EvoformerAttention by @MaxTretikov in https://github.com/deepspeedai/DeepSpeed/pull/8000
• fix: use subprocess instead of os.system in data_analyzer.py by @orbisai0security in https://github.com/deepspeedai/DeepSpeed/pull/7994
• Fix ZeRO-3 forward crash on modules with plain dict _parameters by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/8009
• Remove stale step() docstring from DeepSpeedCPUAdam by @lucaspirola in https://github.com/deepspeedai/DeepSpeed/pull/8011
• Add configurable torch-latest dependency versions by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8016
• Run FastFileWriter fd-close test outside pytest-forked by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8015
• Make GitHub Actions job names unique by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8014
• Support bf16 optimizer states with CPU offload by @lucaspirola in https://github.com/deepspeedai/DeepSpeed/pull/8010
• [fix] fix test_zf.py hang bug by @xbcReal in https://github.com/deepspeedai/DeepSpeed/pull/8012
• [Blog] Muon Optimizer Support in DeepSpeed by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7962
• fix gemma4 num attention head bugs (from [#7975]) by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7990
• fix: add setup_context for torch.func compatibility by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/7916
• Sort and dedupe -gencode flags emitted by op_builder.builder by @adityasingh2400 in https://github.com/deepspeedai/DeepSpeed/pull/8021
• fix(zero): enable vmap on LinearFunctionForZeroStage3 by @roycho96 in https://github.com/deepspeedai/DeepSpeed/pull/8023
• Support flash-attn 2.7.0 in FPDT attention by @xbcReal in https://github.com/deepspeedai/DeepSpeed/pull/8022
• Fix DeepCompile AOT kwargs patching for PyTorch >= v2.11 by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8024

New Contributors

• @Tianyi-Franklin-Wang made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7997
• @jg-heo made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8005
• @MaxTretikov made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8000
• @orbisai0security made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7994
• @lucaspirola made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8011
• @xbcReal made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8012
• @adityasingh2400 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/8021

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.19.0...v0.19.1