-
Notifications
You must be signed in to change notification settings - Fork 3
feat: support finetuning and evaluation of VLA models #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds comprehensive support for finetuning and evaluation of Vision-Language-Action (VLA) models, including OpenPI, OpenVLA, UniVLA, and SmolVLA. The changes introduce a unified CLI interface, model-specific training/evaluation configurations, and extensive infrastructure for model serving and benchmarking.
Key changes:
- Unified CLI interface (
vla-arena train/eval) for all VLA models with dynamic model loading - OpenPI model integration with JAX-based training, Docker deployment, and websocket-based policy serving
- Training and evaluation configurations for OpenPI, OpenVLA (with OFT variant), UniVLA, and SmolVLA
- Removed legacy evaluation utilities and policy implementations in favor of model-specific evaluators
Reviewed changes
Copilot reviewed 124 out of 1401 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| vla_arena/cli/main.py | CLI entry point with train/eval subcommands |
| vla_arena/cli/train.py | Training orchestrator supporting PyTorch (torchrun) and JAX models |
| vla_arena/cli/eval.py | Evaluation launcher for model-specific evaluators |
| vla_arena/models/openpi/* | Complete OpenPI integration including scripts, Docker configs, and client libraries |
| vla_arena/configs/train/*.yaml | Training configurations for all supported models |
| vla_arena/configs/evaluation/*.yaml | Evaluation configurations for all supported models |
| vla_arena/evaluation/* | Removed legacy evaluation utilities and policy base classes |
| tests/* | Updated test structure and fixtures |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
vla_arena/models/openpi/scripts/docker/install_nvidia_container_toolkit.sh
Show resolved
Hide resolved
muchvo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Description
Please include a summary of the changes and which issue is fixed. Include relevant motivation and context.
Fixes # (issue)
Type of change
Please delete options that are not relevant.
Checklist
Go over all the following points, and put an
xin all the boxes that apply.If you are unsure about any of these, don't hesitate to ask. We are here to help!
make format. (required)make lint. (required)make testpass. (required)Testing
Please describe the tests that you ran to verify your changes: