Skip to content

[doc] feat: split GPU/NPU install docs #515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SamitHuang merged 2 commits into from
Sep 7, 2026
+319 −215
Merged

[doc] feat: split GPU/NPU install docs #515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3aa2e5a
[doc] feat: split GPU/NPU install docs
chi030303 Sep 2, 2026
e68e753
[doc] fix: resolve install.md conflicts with main after GPU/NPU split
chi030303 Sep 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Welcome to VeRL-Omni's documentation!

Last updated: 07/20/2026
<!-- 2026-09-02, tianqi, add NPU install page to Getting Started toctree -->
Last updated: 09/02/2026
<!-- end -->

[VeRL-Omni](https://github.com/verl-project/verl-omni) is a general RL training framework focused on multimodal generative models, built on top of [verl](https://github.com/verl-project/verl). It originated from the multi-modal generation RL effort in `verl`, and now has a dedicated home so it can evolve in a more focused way.

Expand All @@ -22,16 +24,19 @@ VeRL-Omni targets RL post-training for three families of generative models:

See {doc}`start/models` for the full model catalogue and which algorithms run on each model.

<!-- 2026-09-02, tianqi, list NPU install next to GPU install in Getting Started -->
```{toctree}
:maxdepth: 2
:caption: Getting Started

start/install.md
start/install_npu.md
start/models.md
start/flowgrpo_quickstart.md
start/multi_node_training.md
start/metrics.md
```
<!-- end -->

```{toctree}
:maxdepth: 1
Expand Down
8 changes: 6 additions & 2 deletions docs/start/flowgrpo_quickstart_npu.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Quickstart: FlowGRPO training on Qwen-Image OCR dataset with Ascend NPU

Last updated: 05/09/2026
<!-- 2026-09-02, tianqi, point NPU quickstart install link at install_npu.md -->
Last updated: 09/02/2026
<!-- end -->

Post-train a diffusion image generation model with FlowGRPO on Atlas 800T A2.

Expand All @@ -14,7 +16,9 @@ Prepare an Atlas 800T A2 server with 8 NPUs, and install the necessary software

1. Install CANN by following the [Ascend CANN installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/900/softwareinst/instg/instg_0003.html?OS=openEuler&InstallType=local).

2. Install VeRL-Omni and its dependencies as described in the [installation guide](install.md#install).
<!-- 2026-09-02, tianqi, point NPU quickstart at the NPU install page -->
2. Install VeRL-Omni and its dependencies as described in the [NPU installation guide](install_npu.md).
<!-- end -->

3. Install the FlowGRPO-specific reward dependency:

Expand Down
225 changes: 13 additions & 212 deletions docs/start/install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,15 @@
# Installation
<!-- 2026-09-06, tianqi, resolve merge conflicts with main: keep GPU-only page; take Python>=3.11, vllm 0.28 extras, and FA fail-fast from main -->
# Installation (NVIDIA GPU)

Last updated: 09/02/2026
Last updated: 09/06/2026

## Requirements
For Ascend NPU, see the {doc}`NPU installation guide <install_npu>`.

For NVIDIA GPU:
## Requirements

* **Python**: Version >= 3.11
* **CUDA**: Version >= 12.8

For Ascend NPU:

* **Python**: Version >= 3.11
* **CANN**: Version == 9.1.0

## Install

```bash
Expand All @@ -30,21 +26,12 @@ source .venv/bin/activate

2. Install the platform backend

For NVIDIA GPU:

```bash
uv pip install -e ".[gpu]" --torch-backend=auto
```

This installs `vllm` for the CUDA PyTorch stack and `kernels` for FA3 backend.

For Ascend NPU:

```bash
uv pip install vllm==0.28.0
uv pip install "vllm-ascend @ git+https://github.com/vllm-project/vllm-ascend.git@$(cat .github/vllm_ascend_pin.txt)"
```

3. Install vLLM-Omni and VeRL-Omni

```bash
Expand All @@ -54,15 +41,6 @@ uv pip install -e ".[train]"

This installs `vllm-omni`, then `verl` and `verl-omni`.

> **Ascend PyTorch version alignment:** VeRL-Omni does not require every NPU
> environment to use one fixed `torch` version such as 2.10.0. Choose a
> mutually compatible `torch` / `torch-npu` pair for the installed CANN and
> vLLM-Ascend versions, and pin that pair before installing the engine and
> training stack. Packages such as `vllm`, `vllm-ascend`, `vllm-omni`, and
> `verl` may resolve different PyTorch versions while they are installed.
> Re-apply the selected pair after all four packages are installed if the
> resolver changed it, then run the version checks below.

### Extras

| Extra | Adds | When |
Expand Down Expand Up @@ -95,7 +73,7 @@ actor_rollout_ref.rollout.rollout_attn_backend=FLASH_ATTN_3_HUB
`FLASH_ATTN_3_HUB` is provided by vLLM-Omni (`kernels-community/flash-attn3`). The legacy
`FLASH_ATTN` rollout path still uses local FA packages (`fa3-fwd` / `flash-attn`).

If FA deps are missing or broken at runtime, requesting an FA2/FA3 backend fails fast instead of silently downgrading to native/SDPA. Fix the install or select `native` / `TORCH_SDPA` explicitly. NPU recipes override with `actor_rollout_ref.model.attn_backend=_native_npu`.
If FA deps are missing or broken at runtime, requesting an FA2/FA3 backend fails fast instead of silently downgrading to native/SDPA. Fix the install or select `native` / `TORCH_SDPA` explicitly.

On older GPUs, prefer FA2 over the FA3 default — both use the same `kernels` Hub path, so nothing extra to install:

Expand Down Expand Up @@ -134,8 +112,6 @@ VeOmni's torch pin has not been validated against torch 2.13 yet — the `--no-d

## Post-Installation Verification

For NVIDIA GPU:

```bash
python -c "import torch; print('torch', torch.__version__, '| CUDA', torch.version.cuda)"
python -c "import vllm; print('vllm', vllm.__version__)"
Expand All @@ -144,39 +120,16 @@ python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"
```

For Ascend NPU:

```bash
python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())"
python -c "import vllm; print('vllm', vllm.__version__)"
python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))"
python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))"
python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"
```

## Build Your Own Docker Image

The repository provides Dockerfiles for both NVIDIA GPU and Ascend NPU environments:

* CUDA Dockerfile: [`docker/Dockerfile.cuda`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.cuda)
* Ascend Atlas A2 NPU Dockerfile: [`docker/Dockerfile.a2.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a2.npu)
* Ascend Atlas A3 NPU Dockerfile: [`docker/Dockerfile.a3.npu`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.a3.npu)
CUDA Dockerfile: [`docker/Dockerfile.cuda`](https://github.com/verl-project/verl-omni/blob/main/docker/Dockerfile.cuda)

The CUDA image is intended for NVIDIA GPU training and rollout. The default CUDA base image uses **CUDA 13.0.2** on Ubuntu 22.04. You can override the CUDA version with `--build-arg CUDA_VERSION=...` if needed.

The NPU images are split by Ascend hardware generation: `Dockerfile.a2.npu` is intended for Ascend 910B / Atlas A2, and `Dockerfile.a3.npu` is intended for Ascend Atlas A3. Both NPU images include CANN, `torch-npu`, `vllm-ascend`, and `vllm-omni`.

The `torch` and `torch-npu` versions in these Dockerfiles are the currently
validated image defaults, not a universal VeRL-Omni requirement. Each
Dockerfile installs the selected pair before the engine stack, then re-applies
the same pair after installing `vllm`, `vllm-ascend`, `vllm-omni`, and `verl`.
This prevents their dependency resolvers from leaving the final image with a
mixed PyTorch stack. When changing PyTorch versions, update both alignment
steps together and keep the pair compatible with the image's CANN version.

Build context is controlled by the repo-root [`.dockerignore`](https://github.com/verl-project/verl-omni/blob/main/.dockerignore); keep large local folders such as `.venv`, `data/`, and `checkpoints/` out of the context.

Ascend NPU images are documented in the {doc}`NPU installation guide <install_npu>`.

## CUDA Docker Image

### Prerequisites
Expand Down Expand Up @@ -231,156 +184,15 @@ Notes:
* **`WORKSPACE`** — example scripts read datasets and write checkpoints under this path. The default is `$HOME` inside the container, i.e. `/root` unless overridden.
* **Hugging Face cache** — mounting `~/.cache/huggingface` avoids re-downloading `Qwen/Qwen-Image` and reward models on every run.

## Ascend NPU Docker Image

### Prerequisites

The Ascend NPU Docker image expects the host machine to provide the Ascend driver and device files.

Before launching the container, make sure the host has:

* Ascend driver installed.
* CANN-compatible runtime environment.
* `npu-smi` available on the host.
* Ascend device nodes under `/dev`, such as `/dev/davinci0`, `/dev/davinci_manager`, `/dev/devmm_svm`, and `/dev/hisi_hdc`.
* Docker permission to pass NPU devices into the container.

The NPU container mounts the host driver directory:

```bash
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro
```

This allows the containerized CANN / `torch-npu` runtime to use the host Ascend driver.

### Build commands

From the repository root, choose the Dockerfile that matches your Ascend hardware.

For Ascend Atlas A3:

```bash
docker build \
-f docker/Dockerfile.a3.npu \
-t verl-omni:npu-a3 \
.
```

For Ascend Atlas A2 / 910B:

```bash
docker build \
-f docker/Dockerfile.a2.npu \
-t verl-omni:npu-a2 \
.
```

When debugging dependency installation or making sure no old Docker layer is reused, add `--no-cache`:

```bash
# Atlas A3
docker build --no-cache \
-f docker/Dockerfile.a3.npu \
-t verl-omni:npu-a3 \
.

# Atlas A2 / 910B
docker build --no-cache \
-f docker/Dockerfile.a2.npu \
-t verl-omni:npu-a2 \
.
```

You may choose different image tags locally. If you do so, replace the image name in the `docker run` command accordingly.

### Launch on Ascend Atlas A3, 16 NPU

Use this command on a 16-card Ascend Atlas A3 machine:

```bash
DEVICES=""
for i in $(seq 0 15); do
DEVICES="$DEVICES --device=/dev/davinci$i"
done

docker run -it --rm \
--name verl_omni_16npu \
--network host \
--ipc host \
$DEVICES \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /mnt/data:/mnt/data \
verl-omni:npu-a3 \
bash
```

### Launch on Ascend Atlas A2 / 910B, 8 NPU

Use this command on an 8-card Ascend Atlas A2 / 910B machine:

```bash
DEVICES=""
for i in $(seq 0 7); do
DEVICES="$DEVICES --device=/dev/davinci$i"
done

docker run -it --rm \
--name verl_omni_8npu \
--network host \
--ipc host \
$DEVICES \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
-v /home:/home \
verl-omni:npu-a2 \
bash
```

### Notes for NPU containers

* **`--network host`** — useful for Ray, distributed training, and multi-process communication.
* **`--ipc host`** — avoids shared-memory limitations during training and rollout.
* **`/dev/davinci*` devices** — expose Ascend NPU cards to the container.
* **`/dev/davinci_manager`**, **`/dev/devmm_svm`**, and **`/dev/hisi_hdc`** — required Ascend runtime device files.
* **`/usr/local/Ascend/driver`** — mounted read-only from the host so the container can use the installed Ascend driver.
* **`npu-smi`** — mounted from the host to inspect device status inside the container.
* **Atlas A3 16 NPU** — exposes `/dev/davinci0` through `/dev/davinci15`.
* **Atlas A2 / 910B 8 NPU** — exposes `/dev/davinci0` through `/dev/davinci7`.

Inside the container, confirm the NPU environment:

```bash
npu-smi info
python -c "from importlib.metadata import version; import torch, torch_npu; print('torch', torch.__version__, '| torch-npu', version('torch-npu'), '| NPU', torch.npu.is_available())"
python -c "import vllm; print('vllm', vllm.__version__)"
python -c "from importlib.metadata import version; import vllm_ascend; print('vllm-ascend', version('vllm-ascend'))"
python -c "from importlib.metadata import version; import vllm_omni; print('vllm-omni', version('vllm-omni'))"
python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"
```

## Example: Qwen-Image FlowGRPO training in Docker

This walkthrough follows the [FlowGRPO quickstart](flowgrpo_quickstart.md) using the OCR dataset and `examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh`.

For CUDA, use the **`ocr` image target** (`verl-omni:gpu-ocr`) so the `Levenshtein` dependency is present.

For Ascend NPU, use the NPU image and the NPU-specific recipe options. NPU recipes should override the attention backend with:

```bash
actor_rollout_ref.model.attn_backend=_native_npu
```
Use the **`ocr` image target** (`verl-omni:gpu-ocr`) so the `Levenshtein` dependency is present.

### 1. Launch the interactive container

Use either the CUDA or NPU launch command above.
Use the CUDA launch command above.

### 2. Prepare the OCR dataset inside the container

Expand All @@ -405,26 +217,15 @@ export WANDB_API_KEY=<your_wandb_api_key>

### 4. Run FlowGRPO training

For CUDA, the default OCR LoRA script uses 4 GPUs by default:
The default OCR LoRA script uses 4 GPUs by default:

```bash
bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh
```

For Ascend NPU, use the corresponding NPU recipe script if available in your checkout:

```bash
bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_npu.sh
```

If you run the training command manually, make sure to include the NPU-specific Hydra override:

```bash
actor_rollout_ref.model.attn_backend=_native_npu
```

The script launches `python3 -m verl_omni.trainer.main_diffusion` with FlowGRPO + `vllm_omni` rollout and OCR reward (`compute_score_ocr`). Checkpoints are written to:

```bash
checkpoints/flow_grpo/qwen_image_ocr_lora
```
<!-- end -->
Loading