diff --git a/docs/index.md b/docs/index.md index fca0839a9..bc97f7d43 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,8 @@ # Welcome to VeRL-Omni's documentation! -Last updated: 07/20/2026 + +Last updated: 09/02/2026 + [VeRL-Omni](https://github.com/verl-project/verl-omni) is a general RL training framework focused on multimodal generative models, built on top of [verl](https://github.com/verl-project/verl). It originated from the multi-modal generation RL effort in `verl`, and now has a dedicated home so it can evolve in a more focused way. @@ -22,16 +24,19 @@ VeRL-Omni targets RL post-training for three families of generative models: See {doc}`start/models` for the full model catalogue and which algorithms run on each model. + ```{toctree} :maxdepth: 2 :caption: Getting Started start/install.md +start/install_npu.md start/models.md start/flowgrpo_quickstart.md start/multi_node_training.md start/metrics.md ``` + ```{toctree} :maxdepth: 1 diff --git a/docs/start/flowgrpo_quickstart_npu.md b/docs/start/flowgrpo_quickstart_npu.md index add0d6576..90a2c1dd9 100644 --- a/docs/start/flowgrpo_quickstart_npu.md +++ b/docs/start/flowgrpo_quickstart_npu.md @@ -1,6 +1,8 @@ # Quickstart: FlowGRPO training on Qwen-Image OCR dataset with Ascend NPU -Last updated: 05/09/2026 + +Last updated: 09/02/2026 + Post-train a diffusion image generation model with FlowGRPO on Atlas 800T A2. @@ -14,7 +16,9 @@ Prepare an Atlas 800T A2 server with 8 NPUs, and install the necessary software 1. Install CANN by following the [Ascend CANN installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/900/softwareinst/instg/instg_0003.html?OS=openEuler&InstallType=local). -2. Install VeRL-Omni and its dependencies as described in the [installation guide](install.md#install). + +2. Install VeRL-Omni and its dependencies as described in the [NPU installation guide](install_npu.md). + 3. Install the FlowGRPO-specific reward dependency: diff --git a/docs/start/install.md b/docs/start/install.md index f870369e0..dfe5fd7ac 100644 --- a/docs/start/install.md +++ b/docs/start/install.md @@ -1,19 +1,15 @@ -# Installation + +# Installation (NVIDIA GPU) -Last updated: 09/02/2026 +Last updated: 09/06/2026 -## Requirements +For Ascend NPU, see the {doc}`NPU installation guide `. -For NVIDIA GPU: +## Requirements * **Python**: Version >= 3.11 * **CUDA**: Version >= 12.8 -For Ascend NPU: - -* **Python**: Version >= 3.11 -* **CANN**: Version == 9.1.0 - ## Install ```bash @@ -30,21 +26,12 @@ source .venv/bin/activate 2. Install the platform backend -For NVIDIA GPU: - ```bash uv pip install -e ".[gpu]" --torch-backend=auto ``` This installs `vllm` for the CUDA PyTorch stack and `kernels` for FA3 backend. -For Ascend NPU: - -```bash -uv pip install vllm==0.28.0 -uv pip install "vllm-ascend @ git+https://github.com/vllm-project/vllm-ascend.git@$(cat .github/vllm_ascend_pin.txt)" -``` - 3. Install vLLM-Omni and VeRL-Omni ```bash @@ -54,15 +41,6 @@ uv pip install -e ".[train]" This installs `vllm-omni`, then `verl` and `verl-omni`. -> **Ascend PyTorch version alignment:** VeRL-Omni does not require every NPU -> environment to use one fixed `torch` version such as 2.10.0. Choose a -> mutually compatible `torch` / `torch-npu` pair for the installed CANN and -> vLLM-Ascend versions, and pin that pair before installing the engine and -> training stack. Packages such as `vllm`, `vllm-ascend`, `vllm-omni`, and -> `verl` may resolve different PyTorch versions while they are installed. -> Re-apply the selected pair after all four packages are installed if the -> resolver changed it, then run the version checks below. - ### Extras | Extra | Adds | When | @@ -95,7 +73,7 @@ actor_rollout_ref.rollout.rollout_attn_backend=FLASH_ATTN_3_HUB `FLASH_ATTN_3_HUB` is provided by vLLM-Omni (`kernels-community/flash-attn3`). The legacy `FLASH_ATTN` rollout path still uses local FA packages (`fa3-fwd` / `flash-attn`). -If FA deps are missing or broken at runtime, requesting an FA2/FA3 backend fails fast instead of silently downgrading to native/SDPA. Fix the install or select `native` / `TORCH_SDPA` explicitly. NPU recipes override with `actor_rollout_ref.model.attn_backend=_native_npu`. +If FA deps are missing or broken at runtime, requesting an FA2/FA3 backend fails fast instead of silently downgrading to native/SDPA. Fix the install or select `native` / `TORCH_SDPA` explicitly. On older GPUs, prefer FA2 over the FA3 default — both use the same `kernels` Hub path, so nothing extra to install: @@ -134,8 +112,6 @@ VeOmni's torch pin has not been validated against torch 2.13 yet — the `--no-d ## Post-Installation Verification -For NVIDIA GPU: - ```bash python -c "import torch; print('torch', torch.__version__, '| CUDA', torch.version.cuda)" python -c "import vllm; print('vllm', vllm.__version__)" @@ -144,39 +120,16 @@ python -c "import verl; print('verl', verl.__version__)" python -c "import verl_omni; print('VeRL-Omni ready')" ``` -For Ascend NPU: - -```bash -python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())" -python -c "import vllm; print('vllm', vllm.__version__)" -python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))" -python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))" -python -c "import verl; print('verl', verl.__version__)" -python -c "import verl_omni; print('VeRL-Omni ready')" -``` - ## Build Your Own Docker Image -The repository provides Dockerfiles for both NVIDIA GPU and Ascend NPU environments: - -* CUDA Dockerfile: [`docker/Dockerfile.cuda`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.cuda) -* Ascend Atlas A2 NPU Dockerfile: [`docker/Dockerfile.a2.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a2.npu) -* Ascend Atlas A3 NPU Dockerfile: [`docker/Dockerfile.a3.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a3.npu) +CUDA Dockerfile: [`docker/Dockerfile.cuda`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.cuda) The CUDA image is intended for NVIDIA GPU training and rollout. The default CUDA base image uses **CUDA 13.0.2** on Ubuntu 22.04. You can override the CUDA version with `--build-arg CUDA_VERSION=...` if needed. -The NPU images are split by Ascend hardware generation: `Dockerfile.a2.npu` is intended for Ascend 910B / Atlas A2, and `Dockerfile.a3.npu` is intended for Ascend Atlas A3. Both NPU images include CANN, `torch-npu`, `vllm-ascend`, and `vllm-omni`. - -The `torch` and `torch-npu` versions in these Dockerfiles are the currently -validated image defaults, not a universal VeRL-Omni requirement. Each -Dockerfile installs the selected pair before the engine stack, then re-applies -the same pair after installing `vllm`, `vllm-ascend`, `vllm-omni`, and `verl`. -This prevents their dependency resolvers from leaving the final image with a -mixed PyTorch stack. When changing PyTorch versions, update both alignment -steps together and keep the pair compatible with the image's CANN version. - Build context is controlled by the repo-root [`.dockerignore`](https://github.com/verl-project/verl-omni/blob/main/.dockerignore); keep large local folders such as `.venv`, `data/`, and `checkpoints/` out of the context. +Ascend NPU images are documented in the {doc}`NPU installation guide `. + ## CUDA Docker Image ### Prerequisites @@ -231,156 +184,15 @@ Notes: * **`WORKSPACE`** — example scripts read datasets and write checkpoints under this path. The default is `$HOME` inside the container, i.e. `/root` unless overridden. * **Hugging Face cache** — mounting `~/.cache/huggingface` avoids re-downloading `Qwen/Qwen-Image` and reward models on every run. -## Ascend NPU Docker Image - -### Prerequisites - -The Ascend NPU Docker image expects the host machine to provide the Ascend driver and device files. - -Before launching the container, make sure the host has: - -* Ascend driver installed. -* CANN-compatible runtime environment. -* `npu-smi` available on the host. -* Ascend device nodes under `/dev`, such as `/dev/davinci0`, `/dev/davinci_manager`, `/dev/devmm_svm`, and `/dev/hisi_hdc`. -* Docker permission to pass NPU devices into the container. - -The NPU container mounts the host driver directory: - -```bash --v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro -``` - -This allows the containerized CANN / `torch-npu` runtime to use the host Ascend driver. - -### Build commands - -From the repository root, choose the Dockerfile that matches your Ascend hardware. - -For Ascend Atlas A3: - -```bash -docker build \ - -f docker/Dockerfile.a3.npu \ - -t verl-omni:npu-a3 \ - . -``` - -For Ascend Atlas A2 / 910B: - -```bash -docker build \ - -f docker/Dockerfile.a2.npu \ - -t verl-omni:npu-a2 \ - . -``` - -When debugging dependency installation or making sure no old Docker layer is reused, add `--no-cache`: - -```bash -# Atlas A3 -docker build --no-cache \ - -f docker/Dockerfile.a3.npu \ - -t verl-omni:npu-a3 \ - . - -# Atlas A2 / 910B -docker build --no-cache \ - -f docker/Dockerfile.a2.npu \ - -t verl-omni:npu-a2 \ - . -``` - -You may choose different image tags locally. If you do so, replace the image name in the `docker run` command accordingly. - -### Launch on Ascend Atlas A3, 16 NPU - -Use this command on a 16-card Ascend Atlas A3 machine: - -```bash -DEVICES="" -for i in $(seq 0 15); do - DEVICES="$DEVICES --device=/dev/davinci$i" -done - -docker run -it --rm \ - --name verl_omni_16npu \ - --network host \ - --ipc host \ - $DEVICES \ - --device=/dev/davinci_manager \ - --device=/dev/devmm_svm \ - --device=/dev/hisi_hdc \ - -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \ - -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \ - -v /mnt/data:/mnt/data \ - verl-omni:npu-a3 \ - bash -``` - -### Launch on Ascend Atlas A2 / 910B, 8 NPU - -Use this command on an 8-card Ascend Atlas A2 / 910B machine: - -```bash -DEVICES="" -for i in $(seq 0 7); do - DEVICES="$DEVICES --device=/dev/davinci$i" -done - -docker run -it --rm \ - --name verl_omni_8npu \ - --network host \ - --ipc host \ - $DEVICES \ - --device=/dev/davinci_manager \ - --device=/dev/devmm_svm \ - --device=/dev/hisi_hdc \ - -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \ - -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \ - -v /home:/home \ - verl-omni:npu-a2 \ - bash -``` - -### Notes for NPU containers - -* **`--network host`** — useful for Ray, distributed training, and multi-process communication. -* **`--ipc host`** — avoids shared-memory limitations during training and rollout. -* **`/dev/davinci*` devices** — expose Ascend NPU cards to the container. -* **`/dev/davinci_manager`**, **`/dev/devmm_svm`**, and **`/dev/hisi_hdc`** — required Ascend runtime device files. -* **`/usr/local/Ascend/driver`** — mounted read-only from the host so the container can use the installed Ascend driver. -* **`npu-smi`** — mounted from the host to inspect device status inside the container. -* **Atlas A3 16 NPU** — exposes `/dev/davinci0` through `/dev/davinci15`. -* **Atlas A2 / 910B 8 NPU** — exposes `/dev/davinci0` through `/dev/davinci7`. - -Inside the container, confirm the NPU environment: - -```bash -npu-smi info -python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())" -python -c "import vllm; print('vllm', vllm.__version__)" -python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))" -python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))" -python -c "import verl; print('verl', verl.__version__)" -python -c "import verl_omni; print('VeRL-Omni ready')" -``` - ## Example: Qwen-Image FlowGRPO training in Docker This walkthrough follows the [FlowGRPO quickstart](flowgrpo_quickstart.md) using the OCR dataset and `examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh`. -For CUDA, use the **`ocr` image target** (`verl-omni:gpu-ocr`) so the `Levenshtein` dependency is present. - -For Ascend NPU, use the NPU image and the NPU-specific recipe options. NPU recipes should override the attention backend with: - -```bash -actor_rollout_ref.model.attn_backend=_native_npu -``` +Use the **`ocr` image target** (`verl-omni:gpu-ocr`) so the `Levenshtein` dependency is present. ### 1. Launch the interactive container -Use either the CUDA or NPU launch command above. +Use the CUDA launch command above. ### 2. Prepare the OCR dataset inside the container @@ -405,26 +217,15 @@ export WANDB_API_KEY= ### 4. Run FlowGRPO training -For CUDA, the default OCR LoRA script uses 4 GPUs by default: +The default OCR LoRA script uses 4 GPUs by default: ```bash bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh ``` -For Ascend NPU, use the corresponding NPU recipe script if available in your checkout: - -```bash -bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_npu.sh -``` - -If you run the training command manually, make sure to include the NPU-specific Hydra override: - -```bash -actor_rollout_ref.model.attn_backend=_native_npu -``` - The script launches `python3 -m verl_omni.trainer.main_diffusion` with FlowGRPO + `vllm_omni` rollout and OCR reward (`compute_score_ocr`). Checkpoints are written to: ```bash checkpoints/flow_grpo/qwen_image_ocr_lora ``` + diff --git a/docs/start/install_npu.md b/docs/start/install_npu.md new file mode 100644 index 000000000..3d7127062 --- /dev/null +++ b/docs/start/install_npu.md @@ -0,0 +1,294 @@ + +# Installation (Ascend NPU) + +Last updated: 09/06/2026 + +For NVIDIA GPU, see the {doc}`GPU installation guide `. + +## Requirements + + +* **Python**: Version >= 3.11 +* **CANN**: Version == 9.1.0 + + +## Install + +```bash +git clone https://github.com/verl-project/verl-omni.git +cd verl-omni +``` + +1. Create a Python virtual environment + +```bash +uv venv --python 3.12 --seed +source .venv/bin/activate +``` + +2. Install the platform backend + + +```bash +uv pip install vllm==0.28.0 +uv pip install "vllm-ascend @ git+https://github.com/vllm-project/vllm-ascend.git@$(cat .github/vllm_ascend_pin.txt)" +``` + + +3. Install vLLM-Omni and VeRL-Omni + +```bash +uv pip install "vllm-omni @ git+https://github.com/vllm-project/vllm-omni.git@$(cat .github/vllm_omni_pin.txt)" +uv pip install -e ".[train]" +``` + +This installs `vllm-omni`, then `verl` and `verl-omni`. + +> **Ascend PyTorch version alignment:** VeRL-Omni does not require every NPU +> environment to use one fixed `torch` version such as 2.10.0. Choose a +> mutually compatible `torch` / `torch-npu` pair for the installed CANN and +> vLLM-Ascend versions, and pin that pair before installing the engine and +> training stack. Packages such as `vllm`, `vllm-ascend`, `vllm-omni`, and +> `verl` may resolve different PyTorch versions while they are installed. +> Re-apply the selected pair after all four packages are installed if the +> resolver changed it, then run the version checks below. + +### Extras + + +| Extra | Adds | When | +| ----------- | ------------------------------------------------------------- | ------------------------ | +| `vllm-omni` | `vllm-omni==0.28.0rc1` | Optional PyPI baseline only; CI/docs use the git pin above | +| `train` | `verl` @ [`.github/verl_pin.txt`](../../.github/verl_pin.txt) | RL training | +| `dev` | `pytest`, `pre-commit`, `Levenshtein`, … | Local development / CI | +| `ocr` | `Levenshtein` | OCR reward (FlowGRPO) | + + +The CUDA `gpu` extra (`vllm`, `kernels`, `liger-kernel`) is not used on NPU. NPU recipes override the attention backend with `actor_rollout_ref.model.attn_backend=_native_npu`. + +## Optional Dependencies + +| Extra | Install | When needed | +| ------------------- | --------------------------------------- | --------------------------------------- | +| OCR reward | `uv pip install -e ".[ocr]"` | FlowGRPO training with OCR-based reward | +| Multimodal training | `pip install qwen-vl-utils math-verify` | Vision-language training (e.g. MMK12) | +| Dev tools | `uv pip install -e ".[dev]"` | Linting and unit tests | + +## Post-Installation Verification + +```bash +python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())" +python -c "import vllm; print('vllm', vllm.__version__)" +python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))" +python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))" +python -c "import verl; print('verl', verl.__version__)" +python -c "import verl_omni; print('VeRL-Omni ready')" +``` + +## Build Your Own Docker Image + +* Ascend Atlas A2 NPU Dockerfile: [`docker/Dockerfile.a2.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a2.npu) +* Ascend Atlas A3 NPU Dockerfile: [`docker/Dockerfile.a3.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a3.npu) + +The NPU images are split by Ascend hardware generation: `Dockerfile.a2.npu` is intended for Ascend 910B / Atlas A2, and `Dockerfile.a3.npu` is intended for Ascend Atlas A3. Both NPU images include CANN, `torch-npu`, `vllm-ascend`, and `vllm-omni`. + +The `torch` and `torch-npu` versions in these Dockerfiles are the currently +validated image defaults, not a universal VeRL-Omni requirement. Each +Dockerfile installs the selected pair before the engine stack, then re-applies +the same pair after installing `vllm`, `vllm-ascend`, `vllm-omni`, and `verl`. +This prevents their dependency resolvers from leaving the final image with a +mixed PyTorch stack. When changing PyTorch versions, update both alignment +steps together and keep the pair compatible with the image's CANN version. + +Build context is controlled by the repo-root [`.dockerignore`](https://github.com/verl-project/verl-omni/blob/main/.dockerignore); keep large local folders such as `.venv`, `data/`, and `checkpoints/` out of the context. + +NVIDIA GPU images are documented in the {doc}`GPU installation guide `. + +## Ascend NPU Docker Image + +### Prerequisites + +The Ascend NPU Docker image expects the host machine to provide the Ascend driver and device files. + +Before launching the container, make sure the host has: + +* Ascend driver installed. +* CANN-compatible runtime environment. +* `npu-smi` available on the host. +* Ascend device nodes under `/dev`, such as `/dev/davinci0`, `/dev/davinci_manager`, `/dev/devmm_svm`, and `/dev/hisi_hdc`. +* Docker permission to pass NPU devices into the container. + +The NPU container mounts the host driver directory: + +```bash +-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro +``` + +This allows the containerized CANN / `torch-npu` runtime to use the host Ascend driver. + +### Build commands + +From the repository root, choose the Dockerfile that matches your Ascend hardware. + +For Ascend Atlas A3: + +```bash +docker build \ + -f docker/Dockerfile.a3.npu \ + -t verl-omni:npu-a3 \ + . +``` + +For Ascend Atlas A2 / 910B: + +```bash +docker build \ + -f docker/Dockerfile.a2.npu \ + -t verl-omni:npu-a2 \ + . +``` + +When debugging dependency installation or making sure no old Docker layer is reused, add `--no-cache`: + +```bash +# Atlas A3 +docker build --no-cache \ + -f docker/Dockerfile.a3.npu \ + -t verl-omni:npu-a3 \ + . + +# Atlas A2 / 910B +docker build --no-cache \ + -f docker/Dockerfile.a2.npu \ + -t verl-omni:npu-a2 \ + . +``` + +You may choose different image tags locally. If you do so, replace the image name in the `docker run` command accordingly. + +### Launch on Ascend Atlas A3, 16 NPU + +Use this command on a 16-card Ascend Atlas A3 machine: + +```bash +DEVICES="" +for i in $(seq 0 15); do + DEVICES="$DEVICES --device=/dev/davinci$i" +done + +docker run -it --rm \ + --name verl_omni_16npu \ + --network host \ + --ipc host \ + $DEVICES \ + --device=/dev/davinci_manager \ + --device=/dev/devmm_svm \ + --device=/dev/hisi_hdc \ + -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \ + -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \ + -v /mnt/data:/mnt/data \ + verl-omni:npu-a3 \ + bash +``` + +### Launch on Ascend Atlas A2 / 910B, 8 NPU + +Use this command on an 8-card Ascend Atlas A2 / 910B machine: + +```bash +DEVICES="" +for i in $(seq 0 7); do + DEVICES="$DEVICES --device=/dev/davinci$i" +done + +docker run -it --rm \ + --name verl_omni_8npu \ + --network host \ + --ipc host \ + $DEVICES \ + --device=/dev/davinci_manager \ + --device=/dev/devmm_svm \ + --device=/dev/hisi_hdc \ + -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \ + -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \ + -v /home:/home \ + verl-omni:npu-a2 \ + bash +``` + +### Notes for NPU containers + +* **`--network host`** — useful for Ray, distributed training, and multi-process communication. +* **`--ipc host`** — avoids shared-memory limitations during training and rollout. +* **`/dev/davinci*` devices** — expose Ascend NPU cards to the container. +* **`/dev/davinci_manager`**, **`/dev/devmm_svm`**, and **`/dev/hisi_hdc`** — required Ascend runtime device files. +* **`/usr/local/Ascend/driver`** — mounted read-only from the host so the container can use the installed Ascend driver. +* **`npu-smi`** — mounted from the host to inspect device status inside the container. +* **Atlas A3 16 NPU** — exposes `/dev/davinci0` through `/dev/davinci15`. +* **Atlas A2 / 910B 8 NPU** — exposes `/dev/davinci0` through `/dev/davinci7`. + +Inside the container, confirm the NPU environment: + +```bash +npu-smi info +python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())" +python -c "import vllm; print('vllm', vllm.__version__)" +python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))" +python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))" +python -c "import verl; print('verl', verl.__version__)" +python -c "import verl_omni; print('VeRL-Omni ready')" +``` + +## Example: Qwen-Image FlowGRPO training in Docker + +This walkthrough follows the [FlowGRPO NPU quickstart](flowgrpo_quickstart_npu.md) using the OCR dataset and `examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_npu.sh`. + +Use the NPU image and the NPU-specific recipe options. NPU recipes should override the attention backend with: + +```bash +actor_rollout_ref.model.attn_backend=_native_npu +``` + +### 1. Launch the interactive container + +Use the Atlas A2 or A3 launch command above. + +### 2. Prepare the OCR dataset inside the container + +```bash +export WORKSPACE=${WORKSPACE:-$HOME} +mkdir -p $WORKSPACE/data/ocr + +# Obtain raw train.txt / test.txt from the Flow-GRPO repo: +# https://github.com/yifan123/flow_grpo/tree/main/dataset/ocr +# Place them under $WORKSPACE/data/ocr/, then preprocess: + +python3 examples/flowgrpo_trainer/data_process/qwenimage_ocr.py \ + --input_dir $WORKSPACE/data/ocr \ + --output_dir $WORKSPACE/data/ocr/qwen_image +``` + +### 3. Optional: Set W&B credentials + +```bash +export WANDB_API_KEY= +``` + +### 4. Run FlowGRPO training + +```bash +bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_npu.sh +``` + +If you run the training command manually, make sure to include the NPU-specific Hydra override: + +```bash +actor_rollout_ref.model.attn_backend=_native_npu +``` + +The script launches `python3 -m verl_omni.trainer.main_diffusion` with FlowGRPO + `vllm_omni` rollout and OCR reward (`compute_score_ocr`). Checkpoints are written to: + +```bash +checkpoints/flow_grpo/qwen_image_ocr_lora +``` +