[examples, gateway, tasks] feat: add DeepEyes training recipe - #150
[examples, gateway, tasks] feat: add DeepEyes training recipe#150Fvvs-iu wants to merge 12 commits into
Conversation
|
The README references examples/deepeyes/run_4b_7p1_container.sh and examples/deepeyes/task_config_4b.yaml, but neither file is included in this PR. Could you add them or update the README to use train_deepeyes.sh directly? |
Thanks for catching this. I’ve updated the README to use train_deepeyes.sh directly and added the Judge startup instructions. |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
6777dae to
7a73617
Compare
| ge=0, | ||
| description="Retries after transient HTTP failures. Set to zero for long-running policy requests.", | ||
| ) | ||
| action_timeout_seconds: float | None = Field(default=None, gt=0.0) |
There was a problem hiding this comment.
Is this variable needed? It seems enforced nowhere.
Summary
Add DeepEyes as a complete visual-tool GRPO training workflow for multimodal
visual question answering.
The DeepEyes policy can inspect image crops through
ImageZoomInToolbeforeproducing a final answer. DeepEyes-specific behavior remains in the owning
layers: the Agent owns the multimodal ReAct loop and crop action; the Task owns
episode lifecycle and reward context; the Task package owns preprocessing and
Judge-based scoring; Gateway and Framework changes preserve compatible
multimodal requests and task metadata during rollout.
The deepeyes recipe also serves as a lightweight, self-contained quickstart
for the agentic RL training path, keeping data preparation, rollout, reward,
and training entry points in one place.
No separate issue is linked because this PR delivers the complete DeepEyes
training recipe and its supporting core integration.
Changes
deepeyesAgent and Task implementations, including theimage-zoom tool, visual-question preprocessing, and Judge-based reward.
examples/deepeyesGRPO recipe: parquet dataset adapter,task/Judge configuration, training launcher, and eight-NPU Qwen3.5-4B preset.
total-token limits; retain the configured system prompt and support
configurable request retries.
preserve task metadata required for reward evaluation.
validation result: Qwen3.5-4B accuracy improves from 50.0% to 79.2%.
Validation
python -m pytest -q tests/uni_agent/agents/test_deepeyes_agent.py5 passedpython -m pytest -q tests/uni_agent/framework/test_generate_sequences_on_cpu.py28 passedpre-commit run --all-files --show-diff-on-failure --color=alwayspassedChecklist
pre-commit run --all-files --show-diff-on-failurepasses.