Add V1 sync recipe for Qwen-Image + MixGRPO - #512
Draft
OnePunchMonk wants to merge 1 commit into
Draft
Conversation
Ports the MixGRPO OCR recipe to the V1 trainer (TransferQueue + ReplayBuffer, sync mode), mirroring the existing SD3.5 FlowGRPO V1 recipe. The sliding-window scheduler's dependency on global_steps carries over unchanged since DiffusionAgentLoopWorkerTQ forwards it into sampling_params the same way the legacy rollout strategy does. Ref verl-project#389
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The updated example README uses a {doc} role that won’t render on GitHub, reducing the usability of the documented instructions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a V1 (TransferQueue + ReplayBuffer) sync-mode MixGRPO recipe for Qwen-Image OCR LoRA training, and documents how to run it alongside the existing V0 recipe and the shared Diffusion V1 setup guide.
Changes:
- Add a new V1 sync-mode launch script for Qwen-Image + MixGRPO (
main_diffusion_v1,trainer.use_v1=true,trainer.v1.trainer_mode=sync). - Update MixGRPO example README to include the V1 recipe and reference the Diffusion V1 setup guide.
- Update MixGRPO algorithm docs to mention the new V1 recipe and link to the Diffusion V1 training guide.
File summaries
| File | Description |
|---|---|
| examples/mixgrpo_trainer/README.md | Documents the new V1 sync recipe and points readers to Diffusion V1 setup guidance. |
| examples/mixgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_mixgrpo_v1.sh | New V1 sync-mode MixGRPO training script for Qwen-Image OCR LoRA. |
| docs/algo/mixgrpo.md | Adds a short note/link to the new V1 recipe and Diffusion V1 training guide. |
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+71
to
+74
| identically to the V0 recipe above. See | ||
| {doc}`Diffusion V1 training <../../docs/start/diffusion_v1>` for setup | ||
| (TransferQueue install, dataset prerequisites) shared with the FlowGRPO V1 | ||
| recipes. |
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a V1 trainer (TransferQueue + ReplayBuffer,
syncmode) recipe for Qwen-Image + MixGRPO, mirroring the existing SD3.5 FlowGRPO V1 recipe (run_sd35_medium_ocr_lora_v1.sh).examples/mixgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_mixgrpo_v1.sh— new V1 sync recipe, usingverl_omni.trainer.main_diffusion_v1withtrainer.use_v1=true trainer.v1.trainer_mode=sync, keeping the same model, LoRA, reward model, and MixGRPO window settings as the V0 recipe.docs/algo/mixgrpo.md,examples/mixgrpo_trainer/README.md— document the V1 script and link to the shareddocs/start/diffusion_v1.mdguide.MixGRPO's sliding-window scheduler (
QwenImageMixGRPOPipelineWithLogProb._maybe_make_progressive_window) depends onglobal_stepsbeing present on the rolloutsampling_params.extra_args. Confirmed this carries over unchanged under V1:DiffusionAgentLoopWorkerTQ.generate_sequencesforwardsglobal_stepsfrom the training batch intosampling_paramsthe same way the legacyvllm_omni_diffusion_strategydoes for V0, and the pipeline adapter dispatch is registered globally byalgorithm=mix_grpo, independent of trainer version.Test plan
bash -non the new scripttotal_training_steps, comparing reward/loss curves (in progress)randomandprogressivesample_strategyboth behave identically to V0 undersyncmodeRef #389