Download Latest Version v0.9.1 source code.zip (3.8 MB) Google Add to Preferred Sources
Home / v0.9.1
Name Modified Size InfoDownloads / Week
Parent folder
README.md 2026-09-20 33.2 kB
0
v0.9.1 source code.tar.gz 2026-09-20 2.7 MB
1 1 weekly downloads
v0.9.1 source code.zip 2026-09-20 3.8 MB
2 2 weekly downloads
Totals: 3 Items   6.6 MB 3

Highlights

Trainer

Unified V1 trainer

  • separate_async can now lend idle trainer GPUs to generation: once a step's prompts are submitted, the trainer keeps its replicas in rollout mode until the replay buffer holds enough sampleable groups, driven by an adaptive starvation threshold. Gated behind trainer.v1.separate_async.enable_switch, off by default (#7373).
  • V0-style fully-async semantics for the V1 async trainers (#7884): actor_rollout_ref.hybrid_engine=False under separate_async (rollout served exclusively by the standalone pool), fractional num_warmup_batches (e.g. 1.5), max_off_policy_threshold=null to disable off-policy version control entirely, and the V0-compatible __num_turns__ metric. All knobs are opt-in and default to the previous behavior.
  • Dynamic micro-batch packing now enforces max_token_len strictly instead of treating ceil(total_len / max_token_len) as the micro-batch count — a bound the Karmarkar-Karp split could legitimately exceed (#7553).
  • DAPO group filtering reads the canonical pre-KL reward from rm_scores and algorithm.filter_groups.metric defaults to reward, so dynamic sampling only needs algorithm.filter_groups.enable=True (#7792).
  • trainer.checkpoint_callback_class names a user-defined CheckpointCallback whose on_save hook fires after every checkpoint save, mirroring the HuggingFace TrainerCallback event — an extension point for uploads, model registries and retention policies without forking the trainer (#7513).
  • Async correctness: skip V1 async warmup after an inflight reissue (#7762), count failed validation sessions in accuracy (#7781), reject requests arriving behind a closed gate instead of parking them (#7912), and keep synthetic padding safe under context parallelism (#7593).
  • verl/experimental/fully_async_policy and verl/experimental/one_step_off_policy are deprecated by V1 trainer and will be moved to verl-recipe in next release.

FSDP

  • Liger Kernel v0.8.2's LigerFusedLinearScaledCrossEntropyFunction replaces the experimental fused linear PPO output head when Liger is installed: 13.5% faster actor updates (0.02803 → 0.02424 ms/token, four H100 seeds) and 5.6% lower actor peak memory (35.68 → 33.70 GiB). Without Liger the existing chunked implementation is unchanged (#7461).
  • Non-blocking FSDP2 model transfers (#7347), configurable deferred gradient sync (#7458), and a fix for the model merger concatenating replicated buffers (#7610).
  • New fsdpturbo backend engine (#7362), with its e2e test moved from GPU to NPU (#7633).
  • Qwen3.5-2B on-policy distillation script (#7605) and an updated Qwen3.5-35B GRPO example at 2K prompt / 6K response (#7678).

Megatron

  • Qwen3.5 GRPO training script for the MindSpeed-Bridge and Megatron backends (#7335).
  • VLM THD input padded to TP/CP alignment before the SP scatter (#7702), plus the NPU TND attention-mask shape fix (#7372).
  • R2 router replay is now model-scoped and opt-in (#7805); the earlier THD-packed variant (#7106) was reverted in [#7786].
  • DDP gradient dtype aligned with optimizer precision (#7485) and sequence-mean loss allowed together with per-token normalization (#7915).
  • The mindspeedllm backend engine was removed (#7374) and the Megatron Lite doc reframed as a Megatron Agent Compose preview (#7537).

VeOmni

  • DeepSeek-V4 QAT bf16 fake-quant training (#7577) and gpt-oss expert parallelism (#7397).
  • async_activation_offload config (#7724) and routed_experts stored as torch.int16 to cut its memory footprint (#7407).

Rollout

vLLM

  • Pluggable router: the load balancer moves out of llm_server.py into verl/workers/rollout/router.py behind a RequestLoadBalancer protocol and a get_router_handle() factory. rollout.router_config_path loads an external router plugin from YAML by router_class FQN with Hydra defaults composition and pkg:// URIs; with no config the built-in sticky-session + least-inflight balancer is unchanged (#7115, [#7677]).
  • Weight-sync drain is now gated on the verl side for DP>1: vLLM's pause_generation stops the scheduler but still queues new adds, so a request landing mid-pause could set engines_running with nothing left to clear it and time out the drain (#7511). Related: scheduler resume race during async weight sync (#7846) and request aborts on multi-node replicas (#7652).
  • Explicit False is preserved for Optional[bool] engine args in CLI serialization (#7508) and for delayed-default boolean flags (#7879).
  • Out-of-vocabulary tokens are masked with a tensor index_fill to avoid sampling errors (#7882); prefix-cache hit counts are surfaced in TokenOutput (#7565); max_num_batched_tokens is raised to max_model_len when chunked prefill is disabled (#7632).
  • LoRA sync index misalignment fixed (#7453), and weights are always resumed before a weight sync (#7434).

SGLang

  • DeepSeek-V4 full weight sync works with an SGLang rollout and a Megatron trainer: verl registers a deepseek_v4 config compatibility wrapper for SGLang-only environments, and the bucketer keeps concatenated destination-parameter pairs together so a byte boundary cannot split them (#7369).
  • Video features are forwarded to SGLang (#7802) and LoRA e2e is fixed (#7413).

CheckpointEngine

  • delta_sharded now has a vLLM consumer (#7227). The format was SGLang-only since [#6974]; a verl vLLM WeightTransferEngine adapter decodes each flush and applies it through vLLM's checkpoint-patch API (vllm-project/vllm#50723), leaving model.load_weights() responsible for name mapping, packed layouts, TP slicing and EP placement.
  • delta_sharded for the TorchTitan engine covering FSDP, FSDP+TP, FSDP+EP, HSDP, HSDP+TP and HSDP+EP. Measured on A800, GSM8K GRPO, disaggregated TorchTitan → SGLang: 10.3x faster weight sync at Qwen3-8B on 32 GPUs (3.34 s vs 34.26 s) and 3.7x on 16 GPUs. FSDP+PP remains unsupported (#7324).
  • Opt-in multi_sender mode for the NCCL engine admits actor rank 0's node-local NVLink peers into the broadcast group as relays, so NCCL can drive more than one NIC per node (#7291).
  • Mooncake completion slots are drained across weight versions (#7764) and the HCCL weight-update error on NPU is fixed (#7488).

Weight-sync performance

  • The full Python GC before rollout weight resume is gone (#7864). It was added in v0.9.0 via aggressive_empty_cache(force_sync=True) and caused a measured throughput regression on colocated weight sync after upgrading from v0.8.0, with no GPU-memory benefit (#7848).
  • The remaining full GC on the colocated weight-refit path is also gone (#7873). Memory snapshots showed gc.collect() in BucketedWeightSender/Receiver._cleanup collected nothing and only cost ~433 ms — long enough to accidentally paper over a CUDA IPC race where the receiver ACKs the final bucket before releasing its mapping. The race is fixed rather than slept through.
  • Ranks are synchronized before resuming the KV cache to avoid OOM on non-leader ranks (#7851).

Agentic RL & multimodal

  • Multimodal Continuous Token (#6804): VLContinuousTokenBuilder extends the CT path to SingleTurnAgentLoop and ToolAgentLoop. Incremental processor calls only expand image placeholders into pad tokens, while AgentLoopWorker re-runs the processor over the full message history once at the end to rebuild pixel_values / image_grid_thw. VL builders compose a VLContinuousTokenMixin with their text-family builder via MRO, so Qwen ChatML newline reinsertion and GLM <|observation|> / <|user|> trimming are inherited unchanged.
  • CT follow-ups: generation prompt fused with the final append group (#7628), assistant role supported in merge/encode context (#7643), and DeepSeek tool appends rendered correctly (#7630).
  • RLHFDataset.process_multi_modal_info offloads the synchronous extractor to the default executor so it no longer blocks the agent-loop event thread (#7562).
  • GLM-4V attention patch ported to the transformers 5 rotary API (#7620); Qwen3.5-VL added to _TEXT_TO_VL_FAMILY for unified checkpoint support (#7433).

Tools & Observability

  • torch profiler gains a post hook (profiler.finish_hook_cmd / finish_hook_ranks), corrected discrete/fused behavior, and global_profiler.relocate_results (#7408). Rollout and training profilers no longer overlap in Trainer V1 (#7722).
  • The torch_memory profiler registers a process-local OOM observer that logs the failed allocation size, call stack and allocator summary and dumps a snapshot under <save_path>/oom_<timestamp>/; memory_snapshot_num_steps retains memory history across several profiled steps (#7770).
  • Agent Loop telemetry is forwarded to RL-Insight (#7448); MLflow runs are finalized on finish so the end time is recorded (#7740).

Environment & Hardware

  • uv integration (#7127): dependency management moves to pyproject.toml + a committed uv.lock covering vLLM, SGLang × FSDP and Megatron, with aarch64 support following in [#7627].
  • transformers upgraded to 5.9.0 across CI and images (#7650, [#7654]).
  • Ascend NPU: GLM-5.2 GRPO recipe (#7788, [#7836]) with vllm-ascend patches (#7779, [#7837]), Megatron 0.18.0 stack (#7604, [#7827]), W4A8 MXFP rollout refit (#7868), modular FusedMoE (#7538), IPC weight transfer by default on A5 (#7728), an Atlas 950 Dockerfile (#7897), and the tutorial tree reorganized into zh/ and en/ (#7563).
  • AMD ROCm: attention cache preserved for CUDA graphs (#7455), shuffled AITER FP8 weights restaged (#7470), PPO trainer CI stabilized and pinned to a single NUMA node (#7377, [#7609]), and a launchable AMD Developer Cloud fully-async DAPO notebook (#7690).
  • NCCL_CUMEM_ENABLE set externally is respected for rollout workers (#7564), non-GPU/NPU devices get a device_available check (#7871), and the Slurm Ray network interface is configurable (#7386).

Breaking Changes

  • Dependency management moved to uv (#7127). Environments are now described by pyproject.toml and the committed uv.lock; uv is the supported way to install verl and its backend extras. aarch64 support followed in [#7627].
  • Multimodal Continuous Token changes the ContinuousTokenBuilder surface (#6804): VL builders are composed from VLContinuousTokenMixin plus a text-family builder, and the agent loop postprocesses multimodal tensors over the full message history.
  • v1 async trainer knobs (#7884): hybrid_engine=False is honored under separate_async, num_warmup_batches accepts fractional values, and max_off_policy_threshold accepts null. Defaults preserve the previous behavior.
  • algorithm.filter_groups.metric now defaults to reward and resolves against the canonical pre-KL rm_scores (#7792).
  • Removed config keys: the never-consumed Megatron grad_offload (#7544), actor.router_replay in favor of the engine config (#7466), and the unused ref router replay config (#7536).
  • The mindspeedllm backend engine was removed (#7374).
  • transformers is pinned to 5.9.0 (#7650).

What's Changed

New Contributors

Full Changelog: https://github.com/verl-project/verl/compare/v0.9.0...v0.9.1

Source: README.md, updated 2026-09-20