A unified cloud-edge post-training framework for vision-language-action policies, built on top of verl.
Large VLA models increasingly place inference and fine-tuning on cloud GPU clusters, while simulators, physical robots, and human operators may run on different machines. verl-vla connects these distributed resources into one post-training system, providing a continuous path from human-in-the-loop data collection through supervised fine-tuning, policy evaluation, and reinforcement learning.
Instead of rebuilding the execution stack for every model, environment, or training algorithm, developers compose reusable workflows on top of a shared distributed runtime and environment loop. Maintained recipes provide reproducible starting points that can be adapted to new policies, simulators, and robot platforms.
Documentation · Quick Start · Framework Overview · Changelog
A workflow defines the end-to-end procedure and moves data and checkpoints
between stages. A trainer advances the selected optimization algorithm.
TrainCluster coordinates the distributed workers that execute model training,
rollout, environment interaction, evaluation, recording, and checkpointing.
Together, these layers keep orchestration, algorithms, and distributed
execution separate while allowing them to share one post-training foundation.
TrainCluster organizes resources by role rather than physical location.
Actor, rollout, and environment workers can run together on one machine or be
placed across multi-node GPU clusters, simulator hosts, and robot-side devices.
Workflows use the same high-level operations for training, rollout, evaluation,
recording, checkpoint management, and weight synchronization regardless of the
deployment topology.
The environment loop publishes observations and accepts controls through a browser, so the operator's keyboard, gamepad, or XR controller does not need to be attached to the machine running the simulator or robot. The same interaction path supports teleoperation, demonstration recording, autonomous rollout, and human intervention.
This browser-based interaction loop is shared across simulators and physical robots:
| Isaac Lab Arena | LIBERO | Piper |
|---|---|---|
 |
 |
 |
Human intervention is particularly important when policies are deployed remotely. Cloud inference produces action chunks that the robot executes locally, while inference and network latency make step-by-step action replacement impractical. Instead, an operator can interrupt autonomous execution, provide an arbitrary-length recovery or correction segment, and then return control to the policy. Policy actions and human corrective actions remain part of one continuous recorded trajectory.
Workflows connect data collection, training, evaluation, and checkpoint lifecycles into complete procedures. Simple workflows reuse the same execution layer for teleoperation, SFT, or evaluation; multi-stage algorithms such as RECAP compose evaluation, trajectory collection, return computation, value training, advantage annotation, and policy updates without introducing a separate training stack. Each maintained recipe combines a verified environment, minimal launcher, configuration, and documentation, with reference results recorded for validated experiments.
| Area | Integrations |
|---|---|
| Models | ACT, Gaussian Actor, Pi0.5, and GR00T N1.6 |
| Environments and robots | LIBERO, Isaac Lab Arena, and Piper |
| Training | SFT, SAC-style off-policy training, TD3+BC, FPO, DSRL, and RECAP |
| Human input | Keyboard, gamepad, and XR controller |
Model adapters preserve upstream-native implementations and Hugging Face checkpoint formats. Environment integrations expose a shared lifecycle and observation-action contract, allowing models, environments, and workflows to evolve independently.
Recipes cover the complete post-training lifecycle from data collection to fine-tuning and reinforcement learning.
| Environment or robot | Recipes |
|---|---|
| LIBERO | Keyboard · Gamepad · XR controller |
| Isaac Lab Arena | XR controller |
| Piper | Keyboard teleoperation and demonstration recording |
| Model | Recipes |
|---|---|
| ACT | Official LIBERO Spatial demonstrations · Self-collected LIBERO Spatial demonstrations |
| Gaussian Actor | LIBERO Spatial task 0 |
| Pi0.5 | LIBERO Spatial |
| GR00T N1.6 | LIBERO Spatial |
Support for additional models, environments, training algorithms, and input devices is under active development.
The following minimal example lets you quickly experience verl-vla's browser-based teleoperation workflow. Clone the repository, complete the verified environment setup, then activate the local environment and start keyboard teleoperation on the first LIBERO Spatial task:
source .venv/bin/activate
vvla-teleop \
cluster.env.env_worker.simulator.libero.task_suite_name=libero_spatial \
cluster.env.env_worker.simulator.libero.task_ids='[0]' \
cluster.env.env_worker.teleop.devices='[keyboard]'Open http://localhost:18000 to view the live
teleoperation dashboard. If LIBERO is running on another machine, replace
localhost with that machine's hostname or IP address.
Follow the keyboard controls shown in the dashboard to operate the robot arm. Press Enter to reset the environment and Ctrl+C in the terminal to stop.
Continue with the full Quick Start to record and replay demonstrations, fine-tune and evaluate an ACT policy, collect optional DAgger intervention data, and improve the policy with a compact, single-batch TD3+BC example. The guide also provides an OSMesa CPU-rendering command for machines without a rendering GPU.
| Guide | What it covers |
|---|---|
| Quick Start | An end-to-end LIBERO workflow from teleoperation to training and evaluation |
| Framework Overview | Architecture, workflows, trainers, TrainCluster, integrations, and resource configuration |
| Data Collection | Environment installation and device-specific teleoperation, recording, and intervention examples |
| Fine-Tuning | Reproducible supervised fine-tuning workflows |
| Reinforcement Learning | Reinforcement learning workflows and examples |
| Troubleshooting | Guidance for diagnosing simulator and distributed execution problems |
We warmly welcome contributions. Valuable improvements of any kind, as well as meaningful and reproducible experiments, can help more people bring embodied models into real-world applications. If you encounter a problem or have an idea for a new model, environment, device, training workflow, or experiment, please open a GitHub issue. See CONTRIBUTING.md for contribution guidelines.
verl-vla is built on verl, extending its distributed training infrastructure for robotics and VLA post-training. We sincerely thank the verl team for their foundational work and continued support for this project.
We are grateful to LeRobot, SimpleVLA-RL, RLinf, DSRL, Giga Models, OpenPI, and Evo-RL for the ideas, implementations, and open-source foundations that helped shape this project. In particular, verl-vla's user-facing data and device APIs are organized with reference to LeRobot's elegant API design.
We owe special thanks to the Isaac Lab Arena and NVIDIA Isaac Lab projects and teams, whose substantial contributions and close support have been instrumental to verl-vla.
verl-vla is licensed under the Apache License 2.0.