verl-project · knlnguyen1802 · Sep 1, 2026 · SamitHuang · Sep 3, 2026
@@ -16,6 +16,7 @@ It originated from the multi-modal generation RL effort in `verl`, and now has a
 
 ## News 🔥
 
+- **[2026-09]** [Wan2.2 DanceGRPO](examples/dancegrpo_trainer/README.md) now defaults to the V1 sync trainer (TransferQueue). The v0 auto-detect launcher is deprecated for CUDA.
 - **[2026-08]** **[MiniMax-H3](examples/diffusionnft_trainer/minimax_h3/README.md)** text-to-audio-video is now supported with DiffusionNFT. See the [T2VA LoRA example](examples/diffusionnft_trainer/minimax_h3/README.md).
 - **[2026-08]** We have released [v0.2.0](https://github.com/verl-project/verl-omni/releases/tag/v0.2.0) for faster diffusion rl and more stable Qwen3-Omni multimodal training.
 - **[2026-08]** [LTX2.3](examples/flowgrpo_trainer/ltx2/README.md) text-to-video+audio model is now supported with FlowGRPO.

@@ -1,6 +1,6 @@
 # Config Explanation
 
-Last updated: 08/23/2026
+Last updated: 09/01/2026
 
 VeRL-Omni builds on [verl](https://github.com/verl-project/verl) and reuses the
 same Hydra config surface for shared RL trainer fields (`data`, FSDP actor /
@@ -110,8 +110,13 @@ standalone rollout GPUs from the Ray cluster.
 `actor_rollout_ref.rollout.agent.num_workers` controls CPU request concurrency; it
 does not allocate rollout GPUs and does not need to match `rollout.n_gpus_per_node`.
 
-On a CUDA Ray cluster, the Wan2.2 auto-device recipe forwards trailing Hydra
-overrides, so the same topology can be launched with the NCCL checkpoint backend:
+This topology is v0-only (`trainer.use_v1=false`). The default CUDA DanceGRPO
+recipe is now the V1 sync launcher (`run_wan22_5b_t2v_hpsv3_v1.sh`); use the
+**deprecated** v0 auto-detect script below when you need
+`actor_rollout_ref.separate`.
+
+On a CUDA Ray cluster, that v0 recipe forwards trailing Hydra overrides, so the
+same topology can be launched with the NCCL checkpoint backend:
 
 ```bash
 bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh \
@@ -315,7 +320,10 @@ These sit on the diffusion trainer YAML (in addition to shared verl trainer fiel
 - `trainer.video_fps`: FPS for videos written to `rollout_data_dir` / `validation_data_dir` and logged to W&B (image runs ignore this).
 - `trainer.rollout_data_save_freq`: Dump train rollout every N steps (`1` = every step, `<= 0` = never).
 - `trainer.rollout_data_max_samples` / `validation_data_max_samples`: Cap samples dumped per train / val run (`null` = all).
-- `trainer.use_v1`: Use the V1 trainer (TransferQueue + ReplayBuffer). When `false`, legacy v0 diffusion trainer.
+- `trainer.use_v1`: Use the V1 trainer (TransferQueue + ReplayBuffer). When `false`,
+  the legacy v0 diffusion trainer. Wan2.2 DanceGRPO on CUDA now defaults to V1
+  via `run_wan22_5b_t2v_hpsv3_v1.sh`; the v0 auto-detect launcher is deprecated
+  for CUDA.
 - `trainer.v1.*`: V1 mode / sampler / async placeholders (`trainer_mode`, `max_off_policy_threshold`, …). See {doc}`../start/diffusion_v1`.
 
 ### `reward` — visual reward manager

@@ -1,11 +1,12 @@
 # Diffusion V1 training
 
-Last updated: 08/25/2026
+Last updated: 09/01/2026
 
 This guide runs the diffusion V1 trainer in synchronous or separate-asynchronous
 mode using the provided Stable Diffusion 3.5 Medium FlowGRPO OCR recipes. The V1
 trainer uses TransferQueue and ReplayBuffer to move rollout trajectories into
-the training loop.
+the training loop. Wan2.2 DanceGRPO on CUDA also defaults to the V1 sync
+recipe; see {doc}`../examples/dancegrpo_trainer`.
 
 The examples support a single-node NVIDIA GPU setup. Sync mode uses two GPUs for
 the colocated actor and rollout plus one reward GPU. Separate-async mode also
@@ -89,6 +90,18 @@ Checkpoints are written by default to:
 checkpoints/flow_grpo/sd35_medium_ocr_lora_v1
 ```
 
+### Wan2.2 DanceGRPO (default CUDA recipe)
+
+Wan2.2 DanceGRPO on CUDA now defaults to the same V1 sync trainer:
+
+```bash
+bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh
+```
+
+See {doc}`../examples/dancegrpo_trainer` for dataset and HPSv3 setup. The
+legacy v0 auto-detect script (`run_wan22_5b_t2v_hpsv3_auto.sh`) is
+**deprecated** for CUDA and remains for NPU.
+
 ## Run V1 separate-async mode
 
 Launch the separate-async recipe:

@@ -1,6 +1,6 @@
 # Supported Models
 
-Last updated: 08/19/2026.
+Last updated: 09/01/2026.
 
 VeRL-Omni supports RL post-training for generative models across image, video,
 audio, and omni modalities. This page catalogues every model with a ready-to-run
@@ -157,7 +157,12 @@ guidance scale 4.0, SDE window 2 over `[0,5]` at noise level 1.2, validation at
 
 | Trainer | Example script | GPU config |
 |---------|---------------|------------|
-| DanceGRPO (HPSv3) | `examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_npu.sh` | 8×NPU (Ascend 800T A2) |
+| DanceGRPO (HPSv3, V1 sync) | `examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh` | 8×GPU |
+| DanceGRPO (HPSv3, v0, NPU) | `examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh` | 16×NPU (Ascend 800T A2) |
+
+The CUDA default is the V1 sync recipe (`main_diffusion_v1`, TransferQueue).
+The v0 auto-detect launcher is **deprecated** for CUDA and remains for NPU
+until a V1 NPU recipe lands.
 
 **Reward model:** HPSv3 (Human Preference Score v3) — local safetensors checkpoint
 placed at `$WORKSPACE/CKPT/HPSv3/HPSv3.safetensors`.

@@ -1,16 +1,24 @@
 # DanceGRPO Trainer
 
-Last updated: 08/07/2026
+Last updated: 09/01/2026
 
 This example shows how to post-train `Wan2.2-TI2V-5B` with DanceGRPO on text-to-video generation tasks. DanceGRPO extends FlowGRPO with a score-based SDE step formulation for improved numerical stability during rollout sampling.
 
-For the base Flow-GRPO setup, see [Examples - FlowGRPO Trainer](https://verl-omni.readthedocs.io/en/latest/examples/flowgrpo_trainer.html). For algorithm details, see [Algorithms - Flow-GRPO](../../docs/algo/flowgrpo.md).
+For the base Flow-GRPO setup, see [Examples - FlowGRPO Trainer](https://verl-omni.readthedocs.io/en/latest/examples/flowgrpo_trainer.html). For algorithm details, see [Algorithms - Flow-GRPO](../../docs/algo/flowgrpo.md). For the V1 trainer (TransferQueue + ReplayBuffer), see [Diffusion V1 training](../../docs/start/diffusion_v1.md).
+
+The **default CUDA recipe** is the V1 sync launcher:
+
+- [`run_wan22_5b_t2v_hpsv3_v1.sh`](wan22/run_wan22_5b_t2v_hpsv3_v1.sh) — **GPU**, **V1 sync** (`verl_omni.trainer.main_diffusion_v1`)
+
+> **Deprecated:** `run_wan22_5b_t2v_hpsv3_auto.sh` is the legacy v0 launcher
+> (`verl_omni.trainer.main_diffusion`). It remains for NPU auto-detect and
+> backward compatibility. New CUDA runs should use the V1 script.
 
 ## Installation
 
 Follow the [installation guide](../../docs/start/install.md) to set up the base environment.
 
-The provided script auto-detects whether NPUs or GPUs are available and configures the run accordingly (16 NPUs or 8 GPUs on a single node).
+The V1 CUDA script targets 8 GPUs on a single node. The deprecated v0 auto-detect script still configures 16 NPUs or 8 GPUs.
 
 ## Prepare the dataset
 
@@ -45,17 +53,26 @@ This produces:
 
 ## Run training
 
-### HPSv3 reward
+### HPSv3 reward (default: V1 sync)
 
 Launch the HPSv3 example from the repository root:
 
 ```bash
-bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh
+bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh
 ```
 
-For CUDA V1 sync (TransferQueue + ReplayBuffer), use `examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh`.
+The V1 script is CUDA-only and selects `PolicyGradientDiffusionTrainerV1Sync` via `trainer.v1.trainer_mode=sync`. TransferQueue is force-enabled inside the runner.
 
-The script auto-detects the device (`npu` via `npu-smi info`, or `gpu` via `nvidia-smi`) and exits with an error if neither is found.
+For Ascend NPU, or to keep the legacy v0 trainer, use the auto-detect script:
+
+```bash
+bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh
+```
+
+> **Deprecated:** the auto-detect script uses the v0 `main_diffusion` trainer.
+> Prefer the V1 launcher on CUDA. The v0 script auto-detects the device
+> (`npu` via `npu-smi info`, or `gpu` via `nvidia-smi`) and exits with an
+> error if neither is found.
 
 #### Configurable environment variables
 
@@ -78,10 +95,10 @@ TRAIN_FILES_PATH=/data/my_train.parquet \
 VAL_FILES_PATH=/data/my_val.parquet \
 MODEL_NAME=/path/to/local/model \
 CUSTOM_REWARD_MODEL_PATH=/path/to/HPSv3.safetensors \
-bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh
+bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh
 ```
 
-The script runs `python3 -m verl_omni.trainer.main_diffusion` with:
+The V1 script runs `python3 -m verl_omni.trainer.main_diffusion_v1` with:
 
 - `algorithm.adv_estimator=dance_grpo`
 - `actor_rollout_ref.model.path=Wan-AI/Wan2.2-TI2V-5B-Diffusers`
@@ -91,9 +108,13 @@ The script runs `python3 -m verl_omni.trainer.main_diffusion` with:
 - `actor_rollout_ref.rollout.algo.noise_level=1.2`
 - `actor_rollout_ref.rollout.algo.sde_window_size=2`
 - `reward.custom_reward_function.name=compute_score_hpsv3`
-- `trainer.n_gpus_per_node=16` (NPU) or `8` (GPU)
+- `trainer.use_v1=true`
+- `trainer.v1.trainer_mode=sync`
+- `trainer.n_gpus_per_node=8`
 - `trainer.total_training_steps=120`
 
+The deprecated v0 auto-detect script still uses `python3 -m verl_omni.trainer.main_diffusion` and sets `trainer.n_gpus_per_node=16` (NPU) or `8` (GPU).
+
 ## SDE variants
 
 DanceGRPO supports three SDE step variants via `actor_rollout_ref.rollout.algo.sde_type`:
@@ -121,10 +142,10 @@ The script sets:
 
 ```bash
 trainer.project_name=dance_grpo
-trainer.experiment_name=wan22_5b_t2v_hpsv3_npu  # or wan22_5b_t2v_hpsv3_gpu
+trainer.experiment_name=wan22_5b_t2v_hpsv3_gpu_v1
 ```
 
-These values are set automatically based on the detected device. Override them by editing the `PROJECT_NAME` and `EXPERIMENT_NAME` variables in the script.
+The V1 script hard-codes that experiment name. The deprecated v0 auto-detect script still sets `wan22_5b_t2v_hpsv3_npu` or `wan22_5b_t2v_hpsv3_gpu` from the detected device. Override them by editing the `PROJECT_NAME` and `EXPERIMENT_NAME` variables in the script.
 
 ### Diffusion-specific metrics
 

@@ -1,4 +1,9 @@
 #!/bin/bash
+# DEPRECATED (CUDA): This v0 launcher uses verl_omni.trainer.main_diffusion.
+# New CUDA runs should use the V1 sync recipe:
+#   bash examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh
+# This script remains for NPU auto-detect and backward compatibility.
+#
 # Wan2.2 LoRA RL with DanceGRPO
 #
 # Model: Wan-AI/Wan2.2-TI2V-5B-Diffusers (text+image-to-video, used in T2V mode)
@@ -9,7 +14,6 @@
 #
 # Reference: https://github.com/XueZeyue/DanceGRPO and https://github.com/verl-project/verl-recipe/blob/main/dance_grpo/dance_grpo_mindspeed_mm/
 #
-set -x
 
 if npu-smi info &>/dev/null; then
     DEVICE="npu"
@@ -20,6 +24,12 @@ else
     exit 1
 fi
 echo "Detected device: $DEVICE"
+if [ "$DEVICE" = "gpu" ]; then
+    echo "WARNING: run_wan22_5b_t2v_hpsv3_auto.sh is the deprecated v0 trainer." >&2
+    echo "For CUDA, use examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_v1.sh instead." >&2
+fi
+
+set -x
 
 if [ "$DEVICE" = "npu" ]; then
     export VERL_DATAPROTO_SERIALIZATION_METHOD=numpy

@@ -5,13 +5,13 @@
 # Algorithm: DanceGRPO (reuses FlowGRPO's advantage estimator and loss)
 # Reward: HPSv3 (Human Preference Score v3) - custom reward model
 #
-# This is the CUDA v1 counterpart of run_wan22_5b_t2v_hpsv3_auto.sh. It uses
+# This is the default CUDA recipe for Wan2.2 DanceGRPO. It uses
 # `verl_omni.trainer.main_diffusion_v1`, which selects
 # `PolicyGradientDiffusionTrainerV1Sync` via `trainer.v1.trainer_mode=sync`.
 # TransferQueue is force-enabled inside the runner, so it does not need to be
-# set on the CLI. NPU auto-detect remains on the v0 script.
+# set on the CLI. NPU auto-detect remains on the deprecated v0 script.
 #
-# Reference (legacy v0 script):
+# Reference (deprecated v0 script):
 # verl-omni/examples/dancegrpo_trainer/wan22/run_wan22_5b_t2v_hpsv3_auto.sh
 # Reference: https://github.com/XueZeyue/DanceGRPO and https://github.com/verl-project/verl-recipe/blob/main/dance_grpo/dance_grpo_mindspeed_mm/
 set -x