Download Latest Version v0.9.0 source code.zip (3.4 MB)
Email in envelope

Get an email when there's a new version of verl

Home / v0.9.0
Name Modified Size InfoDownloads / Week
Parent folder
README.md 2026-08-14 54.5 kB
0
v0.9.0 source code.tar.gz 2026-08-14 2.3 MB
1 1 weekly downloads
v0.9.0 source code.zip 2026-08-14 3.4 MB
0
Totals: 3 Items   5.8 MB 1

v0.9.0

Highlights

Training

Megatron

  • DeepSeek-V4 GRPO end-to-end with Megatron-Bridge actor/ref, vLLM rollout and FP8/MXFP4 weight transfer (#6473), plus a contiguous context-parallel layout (#7221) and CP fixes that make long-context DeepSeek-V4 runnable (#7297).
  • Megatron Lite (mlite) backend for DeepSeek-V4, GLM-5 and Kimi-K2.5/K2.6, with 256-GPU GRPO launchers (#6791, [#7091]).
  • Muon optimizer support via Megatron-Core TensorParallelMuon, with AdamW fallback for non-2D params and an opt-in muon_match_adamw_update_rms scale (#7120).
  • Dynamic context parallel scheduling: each packed micro-batch picks a smaller local CP group when sequences don't need the full CP size (#6555).
  • Megatron chunk entropy (#6446) and fused logprob/entropy migrated from a GPTModel.forward monkey-patch to the Megatron output_processor hook (#6933).
  • Packed-sequence throughput work: BSHD micro-batches padded to the mini-batch max seq_len (#6901) and bucketed packed sequence lengths (#7358).
  • Checkpoint manager rewrite with YAML checkpoint config APIs (#6014); optimizer states and DDP grad buckets aligned with model precision (#6526).

VeOmni

  • DeepSeek-V4 support (#7242) and GPT-OSS weight preservation without expert parallelism (#7243).
  • Weight-refit extra memory reduced from O(E) to O(E/ep_size) by replacing the expert all_gather with sequential intra-EP broadcasts (#6612).
  • pad_to_length bucketing for packed micro-batches, collapsing dynamic shapes so torch.compile / Triton / DeepGEMM autotune cost is paid once instead of every step (#7272).
  • EP-aware sharded delta export for fused expert stacks (#7085), DSD optimizer checkpoint state backfill (#7283), and the missing config surface in veomni.yaml (#6505).

FSDP

  • Per-unit LoRA summon with FSDP1/FSDP2 compatibility and strip-modules support (#6512); save_lora_only checkpoints (#7061).
  • Chunked gather-logsumexp for top-K distillation loss to avoid OOM at long context (#6593) and deferred gradient sync during accumulation (#7095).
  • FSDP2 weight export skips the whole-shard staging round trip (#7005).

Rollout

vLLM

  • Prefill-Decode disaggregated rollout (rollout.name=vllm_pd) routing GRPO rollouts across 1 prefill + N decode engines over NIXL or Mooncake (#6243).
  • Full determinism for vLLM rollout and reward-model inference, so two identical runs produce bitwise-aligned reward curves (#6572).
  • DeepSeek-V4 FP8/MXFP4 linear and MoE weight refit fixes, with the quantized weight-sync paths split per scheme (#7224).
  • Weight-sync restructuring (#7179), non-contiguous weight handling in bucketed transfer (#6919), and MTP drafter weights preserved across hybrid sleep (#6661).

SGLang

  • ROCm backend support out of the box via AITER attention/kernel defaults injected through the platform layer (#6664).
  • FP8 ignored-layer support for Qwen3.x GatedDeltaNet (#6906, [#6915]).
  • LoRA correctness: SGLang stays LoRA-free when model.lora.merge=True (#7234); NCCL buffer race guard in _compact_for_bucket (#7139).

CheckpointEngine

  • New delta_sharded checkpoint engine for disaggregated async training: each actor rank byte-diffs its own shard against a pinned shard-sized CPU snapshot, sparsely gathers only the changed (position, value) pairs, and the rollout side applies them in place through the stock custom-weight-loader hook — no rank ever holds a full-model snapshot (#6974).
  • Measured steady-state update_weights speedups vs full NCCL broadcast (H100, GSM8K GRPO, V1 separate_async, SGLang rollout): 2.4x at Qwen2.5-7B, 1.9x at 32B, 3.1x at 72B.
  • Generalized beyond flat Shard(0): block placements (Shard(k), multi-Shard meshes, manual splits) plus a backend-owned HF export contract (#7144), VeOmni FSDP2+EP (#7085), Megatron-Bridge param mappings covering TP+EP and hybrid-Mamba (#7181), and PP/VPP steady delta export (#7223).
  • Transport hardening: HCCL split_weight_chunks (#7205), NCCL broadcast bucket sizing (#7107), separate magic_recv buffer to prevent weight corruption (#6813), and CUDA stream synchronization on broadcast completion (#7083).

Trainer

Unified V1 trainer

  • New unified trainer abstraction covering sync and async training under verl/trainer/ppo/v1 (#6710), enabled by default in this release (#6823). sync, colocate_async and separate_async now share one control-flow implementation, replay buffer and metric surface.
  • Runnable separate_async trainer (#6790) aligned to the same step granularity as the other modes (#6977), with decoupled PPO support (#7188).
  • Staleness control strategies drop and wait against max_off_policy_threshold (#6778), and a unified replay-buffer eviction/refill matrix for stale, DAPO-filtered and failed rollout groups (#7082).
  • Streaming dataloader (data.gen_batch_size) plus async-trainer checkpoint recovery that restarts pending/running prompts on resume while preserving finished samples (#7037).
  • Colocated reward model for sync / colocate_async (#6818); SkipManager unified across skipping schemes (#6097) and wired into V1 (#6897, [#7032]).
  • Fully-async dynamic resource scheduling: trainer-node GPUs switch between rollout and training via a pluggable scaling policy (#6556).
  • New observability: off-policy metrics (#6736) and rollout MoE load-balance metrics (#6853, [#7075]).

Agentic RL

  • uni-agent is a reference stack for long-horizon agent RL on top of verl — bring any harness (Claude Code, Mini-SWE-Agent, or anything speaking an OpenAI/Anthropic-compatible endpoint) through the Uni-Agent Gateway, and run 1,000+ concurrent stateful sessions.
  • Continuous Token mechanism for multi-turn agentic rollout: a reusable builder layer that keeps token continuity across assistant output, tool/environment feedback and the next generation prompt, with Qwen / MiniMax / GLM boundary handling (#6779). Disabled by default.

Hardware support

  • Platform abstraction layer + plugin-based engine override (#6086): verl.plugin.platform exposes a hardware-agnostic device API, @PlatformRegistry.register(platform=...) registers new platforms, and EngineRegistry lets plugins override engines by (model_type, backend, device) — all without patching the verl source tree.
  • New verl-project/verl-hardware-plugin repository, jointly developed with the FlagOS community, ships reference platform and engine implementations for Cambricon MLU, MetaX, Iluvatar BI-V150, Intel XPU, Enflame GCU and FlagOS. Plugins are auto-discovered through the verl.plugins entry point.
  • AMD ROCm: PlatformROCm backend (#6702), DeepSeek-V4-Flash GRPO on AMD GPUs (#7050), pure-torch fast_hadamard_transform fallback for DSA (#7241), streamlined ROCm Dockerfile and README (#6619), corrected FLOPS tables (#6942), and an MI300 e2e_ppo_trainer CI workflow (#6668).
  • Ascend NPU: per-model NPU patches with fault isolation (#6777), Qwen3.5-122B support (#6582) and long-sequence launcher (#6807), MXFP8 training (#6562), fully-async adaptation for vLLM ≥0.18 and ≥0.19 (#6929, [#6886]), Atlas 950DT A5 installation guides, CANN 9.x images, and multi-node nightly CI (#6876).
  • New How to Extend verl guide covering reward functions, tools, agent loops, engines and platforms (#6825).

Algorithms & Reward

  • DRO losses (#7245) and token-sum loss aggregation (#7197).
  • Critic value loss normalized over the global mini-batch instead of per micro-batch (#6957); distillation loss micro-batch normalization fix (#7225).
  • Deterministic reward for user-defined generative RM paths (#7027) and an optional per-sample compute_score timeout in NaiveRewardManager (#6673).
  • REINFORCE++ carries running_return through observation spans (#7300).

Tools & Observability

  • RL-Insight integration: add rl_insight to trainer.logger and set RL_INSIGHT_SERVER_URL to get training metrics, RL state traces and rollout/TransferQueue subsystem metrics in unified Grafana dashboards — no training-loop changes (#6680, [#7048]).
  • torch profiler re-enabled with clearer output naming (#7099) and partial-token window profiling for rollout (#5760).
  • Gemma4 multimodal GRPO (#6715), Open-R1 multimodal and TinyLLaVA-Video-R1 preprocessing/training scripts (#6849), and a processor hook for multimodal RoPE kwargs (#7236).
  • Security: Qwen3 tool parser no longer allows arbitrary code execution (#6542).

Breaking Changes

  • The V1 PPO trainer (verl/trainer/ppo/v1) is now the default (#6823). separate_async also switches to the same step granularity as the other trainers (#6977).
  • Vanilla mBridge is deprecated; Megatron-Bridge is the default for the Megatron backend (#6951).
  • vLLM older than 0.18.0 is no longer supported; all compatibility branches at or below that version were removed (#7190).
  • Checkpoint engine: the misleading trainer argument is renamed to actor_wg (#6724).
  • The delta custom weight loader moved to verl.workers.rollout.sglang_rollout.delta_loader.apply_delta; update custom_weight_loader config strings (#6974).

What's Changed

New Contributors

Full Changelog: https://github.com/verl-project/verl/compare/v0.8.0...v0.9.0

Source: README.md, updated 2026-08-14