[BREAKING][framework, gateway, tasks] refactor: replace mixed reward info with typed runner results

[zackcxb]

Summary

The previous reward flow used one flat reward_info dictionary for four different contracts: outcome reward, episode completion, validation metrics, and reward-worker context. That made finished look like a metric, forced Framework code to infer semantics by removing reserved keys, and allowed structured scorer context to leak into validation fields.

This PR makes every managed AgentRunner return a typed TaskResult directly and moves reward annotations out of Gateway session state. Gateway continues to own token materialization and session lifecycle fencing; Framework owns reward, completion, metric attachment, and training serialization.

No issue is linked because this refactor follows the repository's existing reward-flow analysis and resolves the unassigned reward-metadata channel. The known non-streaming fallback limitation is documented and intentionally deferred.

Changes

• Framework / runners
  • Add an exported AgentRunner protocol whose successful calls return TaskResult.
  • Return TaskResult from inline and Ray-task runners; reject legacy dictionaries and missing results.
  • Keep TaskResult(reward, accuracy, finished, extra_info) as the single Task/Runner result contract.
  • With streaming RewardLoopWorker handles, pass the complete runner result under extra_info["runner_reward_info"] and let the Worker own final reward and validation metrics.
  • Without streaming handles, retain the runner reward as the Framework fallback.
  • Keep structured extra_info as scorer context; only validated scalar Worker output becomes validation metrics.
• Gateway / trajectories
  • Remove /reward_info, reward_info_url, session reward storage, and Trajectory.reward_info.
  • Add explicit Trajectory.finished, reward_score, and reward_metrics fields.
  • Keep Gateway session lifecycle and token-materialization ownership unchanged.
• Training semantics
  • Make mask_unfinished_episode consume finished is False; None remains trainable.
  • Preserve unfinished rewards in rm_scores, retaining the existing group-baseline behavior.
  • Serialize scalar metrics under extra_fields["reward_extra_info"], so TaskResult.accuracy reaches validation as acc.
• Tasks / examples
  • Preserve AgentRunner / agent_runners naming; framework/task_runner.py remains the concrete Task-backed runner implementation.
  • Preserve the source-prompt binding introduced by [BREAKING][framework, tasks, examples, docs] feat: render task prompts from recipes #134 and the OpenYuanrong reverse-tunnel/runtime timeout behavior from current main.
  • Migrate Claude Code, quickstart, and mini-swe-agent recipes from Gateway reward posting to the typed return plus pass-through compute_score path.
  • Bump debug trajectory JSON records from schema version 1 to 2.
• Deferred
  • The no-streaming-handles training path still writes the runner fallback before the trainer's colocated reward pass. This known dual-source window is documented; the colocated pass remains authoritative for training reward.
  • No independent process-reward implementation; future token-level scores need an explicit sibling channel rather than validation metrics.
  • No online/external HTTP result endpoint; a future authenticated adapter should normalize external results to TaskResult.

Validation

• PYTHONPATH="$PWD:$PWD/verl" python -m pytest -q tests/uni_agent/framework tests/uni_agent/tasks/test_finished_propagation.py tests/uni_agent/agents/test_react_agent.py tests/uni_agent/agents/test_claude_code_agent.py tests/uni_agent/agents/test_mini_swe_agent_agent.py tests/uni_agent/examples/test_claude_code_runner.py tests/uni_agent/gateway/test_debug_launcher.py tests/uni_agent/gateway/test_gateway_actor_on_cpu.py tests/uni_agent/gateway/test_gateway_manager_on_cpu.py tests/uni_agent/gateway/test_session_multiple_chains_on_cpu.py
  • 250 passed, 5 warnings
• Expanded Gateway run: 303 passed, 3 failed, 7 warnings.
  • The three failures are unchanged vLLM/OpenAI SDK dependency failures: installed vLLM imports openai.types.responses.NamespaceTool, which the installed OpenAI SDK does not expose.
  • The representative failure reproduces on a clean upstream/main worktree.
• pre-commit run --all-files --show-diff-on-failure --color=always
  • Ruff, Ruff format, mypy, and compileall passed.
• mini-swe-agent recipe wiring
  • bash -n examples/mini_swe_agent/run_train.sh passed.
  • Confirmed the recipe has no report_reward / use_reward_loop_worker compatibility config and uses uni_agent.framework.task_runner.compute_score.

No GPU training or full sandbox rollout was run; this change is covered by CPU/Ray contract tests and recipe wiring checks.

Compatibility

This is a breaking API change. No runtime compatibility mapping is retained.

Removed APIs and fields:

• SessionHandle.reward_info_url
• POST /sessions/{id}/reward_info
• GatewaySession.set_reward_info / GatewayActor.set_reward_info
• Trajectory.reward_info
• Runner report_reward
• Legacy runner dictionary or missing-result returns

Migration:

from uni_agent.tasks import TaskResult


async def my_runner(*, session, raw_prompt, sample_index, **kwargs):
    # Execute and evaluate one Agent episode.
    return TaskResult(
        reward=score,
        accuracy=accuracy,
        finished=completed,
        extra_info={"case_id": case_id},
    )

Remove runner_kwargs.report_reward and reward-info HTTP posts from custom integrations. Custom AgentRunner implementations must return TaskResult; use TaskResult(reward=None) when they do not provide a runner reward.

Custom code consuming debug trajectories.jsonl must accept schema version 2 and read finished, reward_score, and reward_metrics instead of reward_info.

The verl/ submodule is unchanged.

Evolution ledger

• ledger_update: candidate
• Evidence: PR [BREAKING][framework, gateway, tasks] refactor: replace mixed reward info with typed runner results #109 replaces the Gateway reward-info channel with the typed AgentRunner -> TaskResult -> Framework/RewardLoopWorker contract.

Checklist

• The PR is focused and explains why no issue is linked.
• The title follows the repository format and names the owning layers.
• Tests cover the behavior and the baseline dependency failure is stated.
• User-facing API, config, and workflow changes include documentation and runnable examples.
• Compatibility impact and migration steps are documented.
• Logs, fixtures, and examples contain no credentials or private data.
• pre-commit run --all-files --show-diff-on-failure passes.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 98.73418% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
uni_agent/framework/framework.py	92.85%	3 Missing ⚠️
tests/uni_agent/support.py	75.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!