DeepSpeed Files

What's Changed

• Update version.txt post 0.17.6 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/7572
• DeepCompile ZeRO-3: robust allgather for uneven shards; fix profiling… by @juyterman1000 in https://github.com/deepspeedai/DeepSpeed/pull/7489
• logging: Also set log level of logger handlers by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7576
• Deepcompile: Fix bugs when applying deepcompile to VLA-like models by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7569
• Broadcast fp16 overflow in Z1 by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7580
• Deepcompile: Make size of activation to free configurable by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7582
• SuperOffload Release by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7559
• Include init file for superoffload folder by @nguyen599 in https://github.com/deepspeedai/DeepSpeed/pull/7591
• disables ZeRO checkpoint loading path when stage=0 by @therealnaveenkamal in https://github.com/deepspeedai/DeepSpeed/pull/7586
• Simplify leaf module hook by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7592
• Fix the universal checkpoint issue for stage3 when there are multiple subgroups. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7585
• Change current_device() to current_device_name() by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7600
• Fixed the problem of loading universal checkpoint error in multi-machine mode. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7601
• DeepCompile: Specify tensor aliasing in C++ op schema by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7597
• DeepCompile: Fuse allgather and downcast by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7588
• Add blog for SuperOffload by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7594
• Add venv to .gitignore by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7605
• Handle the case of DeepCompile's enabled but not activated by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7603
• DeepCompile: Fix IPG bucket clearing by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7610
• Minor fix in the SuperOffload blog by @xylian86 in https://github.com/deepspeedai/DeepSpeed/pull/7612
• Fixed the issue that universal checkpoint cannot be loaded for stage3 when world size expansion. by @zhengchenyu in https://github.com/deepspeedai/DeepSpeed/pull/7599
• Fixed save_checkpoint race when consolidating NVMe offloaded tensors by @H1manshu21 in https://github.com/deepspeedai/DeepSpeed/pull/7613
• [wall_clock_breakdown] always log stats when enabled by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7617
• DeepCompile: Use min_cut_rematerialization for partitioning joint graphs by @eternalNight in https://github.com/deepspeedai/DeepSpeed/pull/7609
• Show mismatching values when DeepCompile test fails by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7618
• Improve leaf module interface (enable via config, relax matching criteria, add document, etc.) by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7604
• add print_dist util by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7621
• Super offload blog Chinese version by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7620
• Enable grad scaler for ZeRO-0 + torch.autocast path by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7619
• Blog of zenflow binding study by @delock in https://github.com/deepspeedai/DeepSpeed/pull/7614
• Clarify document of leaf module config by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/7623
• [TiledMLP] moe support by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/7622
• Update email address by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/7624

New Contributors

• @juyterman1000 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7489
• @nguyen599 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7591
• @zhengchenyu made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7585
• @H1manshu21 made their first contribution in https://github.com/deepspeedai/DeepSpeed/pull/7613

Full Changelog: https://github.com/deepspeedai/DeepSpeed/compare/v0.17.6...v0.18.0