DeepSpeed - Browse /v0.19.8 at SourceForge.net

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
README.md	2026-10-19	18.7 kB	0
v0.19.8 Patch Release source code.tar.gz	2026-10-19	213.6 MB	0
v0.19.8 Patch Release source code.zip	2026-10-19	214.9 MB	0
Totals: 3 Items		428.5 MB	0

What's Changed

Deprecate leftovers of compression library by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8535
Update version.txt after 0.19.7 release by @loadams in https://github.com/deepspeedai/DeepSpeed/pull/8555
Deprecate loco zero++ by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8534
Deprecate sparse_gradients by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8504
docs: stop Codex false-alarm on DCO sign-off by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8561
Raise Modal sandbox timeout to absorb full-suite runtime by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8571
Derive the autocast test backend and scope the bf16 gate to NCCL by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8559
Skip heavy CI for docs-only PRs touching root markdown by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8560
Deprecate APEX amp by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8564
Document sub_group_size and correct its default to 1e9 by @shravanibnikam in https://github.com/deepspeedai/DeepSpeed/pull/8567
Deprecate legacy curriculum learning by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8566
[muon] Per-head Newton-Schulz for attention projections by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8384
Upgrade yapf to v0.43.0 to fix EOFError failure in CI by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8574
[AutoTP] Add support for vocab-parallel LM head and related configurations by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8309
Remove Progressive Layer Dropping (PLD) support by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8562
Support Muon in BF16_Optimizer with FP32 gradient accumulation by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8526
Register native pinned host memory with the CANN runtime for NPU by @VenusTZZ in https://github.com/deepspeedai/DeepSpeed/pull/8532
implement register_host_memory and unregister_host_memory for supa by @frozenleaves in https://github.com/deepspeedai/DeepSpeed/pull/8537
Remove elastic training by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8563
Use int64 program ids in the SwiGLU and AutoEP fused restore Triton kernels by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8591
Describe the GPTBigCode and Yuan shared-QK layouts by @Achyuthan-S in https://github.com/deepspeedai/DeepSpeed/pull/8575
Fix HybridEngine OPT generation with modern cache interfaces by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8523
[muon] Per-head Muon for linear-attention layers: read head geometry from the owning module by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8436
[AutoTP] Support eager row-parallel output head training by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8538
Release AutoEP router-logit caches on checkpoint replay exits by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8581
Fix memory leak and NULL-deref in shm_initialize by @tritsystem in https://github.com/deepspeedai/DeepSpeed/pull/8565
Fix silent gradient-bucket drop on CPU in ZeRO-1/2 non-contiguous reduction by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8382
Add opt-in AutoEP Python GC policy by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8451
[AutoTP] Add optional Liger vocabulary-parallel cross entropy by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8540
Overlap AutoEP split planning with token packing by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8359
Normalize Gram Newton-Schulz inputs before casting to fp16 by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8533
Remove the older TiledLoss and UlyssesSPFwdLossBwdWithLogits by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8587
Wait for all async all-gathers in coalesced ZeRO-3 parameter gather by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8539
Skip narrow dtypes in the mixed-precision cast by @sfc-gh-abkulkarni in https://github.com/deepspeedai/DeepSpeed/pull/8416
AutoEP: drop the redundant float32 copy in the default combine by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8621
Remove stale compression test from HPU CI by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8616
Skip fp16 ZeroPP tests on accelerators without fp16 support by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8610
AutoEP: share one DeepEP buffer across MoE layers instead of one per layer by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8557
Implement graph operations for NPU accelerator by @VenusTZZ in https://github.com/deepspeedai/DeepSpeed/pull/8605
Add opt-in regional compilation around AutoEP layers by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8380
Fix pipe tests failures on CPU by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8611
Run Muon once per optimizer step under ZeRO-3, not once per micro-batch by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8600
Fix 2 CPU failures in cpu-torch-latest by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8629
Fix Muon optimizer under ZeRO CPU offload and bound gather buffers by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8464
Loosen the late-iteration gradient tolerance in the checkpointing test by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8648
Add affine AutoEP checkpoint placement by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8544
Fix SequenceTiledCompute backward for empty trailing shards by @taking-lying-flat in https://github.com/deepspeedai/DeepSpeed/pull/8434
fix: Remove the unused ROCm Apex clone, build, and PYTHONPATH setup. by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8617
Avoid repeated full gradient clearing in ZeRO-3 subgroup updates by @aprylewu in https://github.com/deepspeedai/DeepSpeed/pull/8625
Stage Muon's momentum once per partition in ZeRO-1/2 by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8633
A zero-element parameter loses its shape when it is bound to a flat buffer by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8467
Fix Muon aspect-ratio scale truncated when muon_update recompiles by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8641
Deprecate and remove graph harvesting by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8579
Replay AutoEP's cached DeepEP dispatch in the layout it was built with by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8636
Keep ZenFlow Adam step counters in float32 by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8496
[AutoTP] Implement VocabParallelEmbedding and enhance tied model support by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8582
docs: add Simplified Chinese translation (README.zh.md) by @JasonYeYuhe in https://github.com/deepspeedai/DeepSpeed/pull/8498
fix(rollout): reject shared prefill with continuous batching by @modelpath-dev in https://github.com/deepspeedai/DeepSpeed/pull/8583
AutoEP: Add activation registry by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8650
Skip ZeRO DeepCompile backward hooks for AutoSP by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8667
Add DualPipeV by @phoeenniixx in https://github.com/deepspeedai/DeepSpeed/pull/8576
Implement FusedLion, FusedLamb, and Quantizer ops for NPU accelerator by @VenusTZZ in https://github.com/deepspeedai/DeepSpeed/pull/8627
Fix Triton compatibility for FlashAttention and MatMul autotuning by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8679
fix(rollout): forward top-k sampling configuration by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8676
Skip fp16-config tests on accelerators without fp16 support by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8684
Consolidate PRs for docs and typos by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8689
Fix ZeRO-3 Muon all-gather for mixed parameter shapes by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8628
Reject amsgrad in DeepSpeedCPUAdam instead of ignoring it by @vineethsaivs in https://github.com/deepspeedai/DeepSpeed/pull/8694
Fix FPDT K/V gradient buffers under grouped-query attention by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8665
Check AutoSP sequence length and head count divisibility by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8675
Validate the AutoSP SP/DP mesh when registering process groups by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8674
Use expert gradients, not expert weights, in the FP16_Optimizer MoE grad norm by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8635
Restore a shard from the map the layer published by @Achyuthan-S in https://github.com/deepspeedai/DeepSpeed/pull/8622
AutoEP: one program per token in the fused weighted restore forward by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8678
Nightly full-suite run with classified failure triage by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8606
Keep Muon's update at full size under fp16 loss scaling by @alanhuangyoo in https://github.com/deepspeedai/DeepSpeed/pull/8655
Add Fengchun Hua (hipudding) to COMMITTERS.md by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8718
Add opt-in fused AutoEP DeepEP row weighting by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8644
AutoEP: MiniMax-M3 preset by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8651
AutoEP: reorder expert rows with gathers on the collective path by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8671
Fix Ulysses SP registration for string model paths and gloo collectives by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8702
Harden ZeRO-1/2 offload gradient storage lifetime and stream ordering by @jeffra in https://github.com/deepspeedai/DeepSpeed/pull/8632
Give rollout cache fakes the per-layer config Transformers now reads by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8730
Add an opt-in fused Triton RoPE for Hugging Face split-half models by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8663
feat(rollout): align decode fronts and trim dead zones by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8497
Fix autotuner crash when an experiment produces no metric by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/8421
Raise Modal nightly Sandbox timeout to 2h by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8746
Keep invalid group-norm sentinels from becoming a finite 1.0 step by @sfc-gh-truwase in https://github.com/deepspeedai/DeepSpeed/pull/8638
Pin Modal torch-latest to OCI and cover PR open events by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8770
Add AWS capacity fallback for torch-latest by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8778
[BUG] Replace deprecated npu_apply_adam(_w) with torch._fused_adam(w)_ on NPU by @Ginray in https://github.com/deepspeedai/DeepSpeed/pull/8640
Release each BF16 gradient once immediate_grad_update has accumulated it by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8654
Stream AWS fallback logs while commands run by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8783
Assert offload-state removal only when offload leaves the accelerator by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8738
[AutoTP] Support tied embeddings for row-parallel output-head training by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8680
Keep the buffers a model names for fp32 in fp32 under ZeRO-3 by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8699
Fix DataAnalyzer.run_map crashing when specific_threads is set by @Rakshit-gen in https://github.com/deepspeedai/DeepSpeed/pull/8758
feat(rollout): add OPSD response token loss primitives by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8677
Use default fp32 tolerances in AutoTP training tests by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8796
Guard the shm inference all-reduce on the host ISA by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8701
Reduce unnecessary CI work for documentation changes and consolidate agent instructions in AGENTS.md by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8794
Avoid circuler import caused by Hybrid Engine and OPT model by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8737
AutoEP + AutoTP folding under ZeRO-3 by @pengdurice in https://github.com/deepspeedai/DeepSpeed/pull/8670
Add an opt-in fused Triton RMSNorm for Llama-style Hugging Face models by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8645
Log the skipped optimizer step on gradient overflow by @sfc-gh-abkulkarni in https://github.com/deepspeedai/DeepSpeed/pull/8417
[UC] Plan a scale-1 affine shard-to-shard transfer without rebuilding the parameter by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8623
Preserve non-floating buffers in ZeRO checkpoint conversion by @aprylewu in https://github.com/deepspeedai/DeepSpeed/pull/8515
[Bug Fix] Fix CPUAdagrad updates with nonzero weight decay by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8781
[checkpoint] Refuse a map whose pieces write the same shard address by @0z5a in https://github.com/deepspeedai/DeepSpeed/pull/8624
[AutoSP] Honor seq_dim when AutoSP shards inputs by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8720
[AutoSP] Fix tensor truthiness in get_node_shape_meta by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8760
Add an opt-in chunked causal-LM loss that keeps no full-vocabulary FP32 tensor by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8653
[AutoSP] Rebuild query-sharded attention masks before SDPA by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8761
Replay AutoEP's DeepEP dispatch layout in non-reentrant checkpoint recompute by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8731
Add an opt-in fused gate+up grouped GEMM for AutoEP experts by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8733
Add a pr-checklist agent skill and an AI contribution policy by @PKUWZP in https://github.com/deepspeedai/DeepSpeed/pull/8744
AutoEP: do not slice DeepEP buffers that already hold exactly the arrived rows by @yh0903 in https://github.com/deepspeedai/DeepSpeed/pull/8747
Use C++20 for XPU op builders on torch 2.12+ by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8803
[bug fix] ZeRO2: preserve fp32 grad accumulation precision in partition storage by @stas00 in https://github.com/deepspeedai/DeepSpeed/pull/8749
Reduce chunked cross-entropy memory test footprint by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8824
[CI] Run torch-latest only on AWS by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8825
Build CPU optimizer ops without CUDA and fix the CPU Lion SIMD update by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8813
Add a nightly multi-rank CPU suite by @delock in https://github.com/deepspeedai/DeepSpeed/pull/8806
Respect BF16 overflow-check opt-out in ZeRO-1/2 by @XFDG in https://github.com/deepspeedai/DeepSpeed/pull/8818
Speed up import deepspeed by deferring heavy imports and import-time work by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8807
Use C++20 for NPU op builders on torch 2.12+ by @VenusTZZ in https://github.com/deepspeedai/DeepSpeed/pull/8816
[fix] NPU: RoPE correctness, inverse-frequency caching, and KV cache preallocation in the inference op by @VenusTZZ in https://github.com/deepspeedai/DeepSpeed/pull/8776
Release GIL while destroying AIO handles by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8832
Add vocabulary-parallel logprobs, entropy and KL ops by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8791
Describe the CodeGen fused-QKV layout as a block selection by @Achyuthan-S in https://github.com/deepspeedai/DeepSpeed/pull/8706
Add vocabulary-parallel fused linear cross entropy by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8792
[CI] Restore Modal-first torch-latest routing by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8849
Fix TestLionConfigs on the CPU accelerator and run it on every runner by @hwchen2017 in https://github.com/deepspeedai/DeepSpeed/pull/8833
Add deepspeed.comm.pg_utils.is_parallel for optional parallel groups by @jinyouzhi in https://github.com/deepspeedai/DeepSpeed/pull/8815
Release process-group caches before CPU/Gloo worker shutdown by @tohtana in https://github.com/deepspeedai/DeepSpeed/pull/8846
Add KLXPU (KUNLUNXIN XPU) accelerator by @GuCarpenter in https://github.com/deepspeedai/DeepSpeed/pull/8799
Enable CUDA Graph for continuous rollouts by @nathon-lee in https://github.com/deepspeedai/DeepSpeed/pull/8732