[examples] feat: blackbox mini-swe-agent training recipe - #86
Open
zhaizhiqiangA wants to merge 9 commits into
Open
[examples] feat: blackbox mini-swe-agent training recipe#86zhaizhiqiangA wants to merge 9 commits into
zhaizhiqiangA wants to merge 9 commits into
Conversation
add debug use global_steps fill min_global_steps & max_global_steps add infer Add agent framework concurrency diagnostics Stabilize blackbox rollout logging npu config
fix reward bug && npu config
sync npu modify
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
zhaizhiqiangA
force-pushed
the
blackbox-recipe-sync-upstream
branch
from
July 23, 2026 08:24
5e314b1 to
d51e978
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds a blackbox RL training recipe for mini-swe-agent under examples/blackbox_recipes/. The agent runs entirely inside an remote sandbox via a sidecar tool-image mount: the host-side runner creates the sandbox, pipes the task config to the in-sandbox agent over stdin, parses the result from stdout, and evaluates the reward in the same sandbox. The agent reaches the LLM through the gateway via an upstream tunnel, so training is fully "blackbox" — the trainer only sees prompts/responses through the gateway. Training uses the V1 unified trainer (Megatron backend, GRPO, separate_async).
Related work:
[train] feat: add blackbox agent gateway (#25) — the gateway this recipe runs against
Checklist Before Starting
gh pr list --repo verl-project/uni-agent --state open --search "mini-swe-agent"
No pull requests match your search in verl-project/uni-agent
[examples] feat: blackbox mini-swe-agent training recipeTest
A full RL training recipe is not practical to cover in CI, so validation was manual:
API and Usage Example
This PR only adds files under examples/ plus minor internal import-path updates; there are no public API changes.
Design & Code Changes
New recipe — examples/blackbox_recipes/mini_swe_agent/
mini_swe_agent_runner.py — host-side runner. Creates a YRSandbox with the sidecar mounted at /opt/mini-swe-agent, base64-encodes the task config (task text + tunnel-rewritten gateway URL + step limit) and pipes it to run_agent.py via stdin, parses the JSON result from stdout (robust to litellm noise), then evaluates the reward in the same sandbox via SandboxEnvForReward and POSTs reward_info. Sandbox is always cleaned up in finally.
run_agent.py — in-sandbox entrypoint. Builds a LocalEnvironment + LitellmModel (pointed at the gateway tunnel) + DefaultAgent from mini-swe-agent's SWE-bench defaults, runs the task, emits a result JSON.
Dockerfile.mini-swe-agent-tool — self-contained, glibc-portable sidecar image (FROM scratch) so the sandbox base image needs no Python/Node.
dataset.py (SWEBenchDataset) injects verl-standard reward fields; reward.py reuses the uni_agent reward-spec registry to score resolved/unresolved in-env.
config/swe_agent_blackbox_megatron_v1.yaml + scripts/run_train.sh — V1 unified trainer, separate_async by default (4 GPU trainer + 4 GPU rollout on one node), vLLM async rollout, GRPO, Megatron offload.
Checklist Before Submitting
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always