epic: agent harness training compatibility

[cwing-nvidia]

Goal

The built-in agent harnesses shipped for evaluation (tracked in #1042) should also be usable for RL training through Gym with minimum friction — i.e. their rollouts carry everything the training pipeline needs (token IDs, logprobs, correct masking, reconstructable multi-turn trajectories), are verified against a tokenizer/model, and are documented per harness.

This is the training counterpart to the evaluation epic #1042. #1042 ensures a harness can produce scored rollouts; this epic ensures those rollouts are training-grade.

Why this matters

Evaluation only requires a harness to return a response that the verifier can score. Training additionally requires:

• Token IDs + logprobs for every model-generated token in the trajectory
• Correct loss masking — tokens injected by the harness (system prompts, tool results, scaffolding, retries) must be distinguishable from model-sampled tokens so they aren't trained on
• Trajectory reconstruction — multi-turn, tool-calling trajectories must be losslessly reconstructable in the order the model saw them (see Trajectory data contracts are fragmented across model-call capture, agent rollouts, and interchange formats #1822, [epic] Token-level stats during rollout and eval #1483)
• Tokenizer/model alignment — the harness's token stream must round-trip against the training model's tokenizer
• Determinism / production parity — inference during rollout should be consistent with how the harness runs in production

Acceptance Criteria

• Each listed harness produces rollouts with token IDs + logprobs and correct loss masking for model-generated tokens
• Multi-turn / tool-calling trajectories are losslessly reconstructable and round-trip against the training model's tokenizer
• A training-compatibility conformance check exists and passes per harness
• At least one end-to-end training run per harness shows the expected reward/benchmark improvement (e.g. SWE-Bench Verified score increase), per the experiment pattern in Run a model-training compatibility experiment with an agent harness #1569
• Each harness's README documents its training-compatibility status and any caveats

Harnesses

Mirrors the eval matrix in #1042. Status here reflects training readiness, not eval.

P0

• Claude Code
• Codex
• Hermes
• OpenClaw
• OpenCode
• OpenHands
• Mini-SWE-Agent
• KiloCode
• Terminus

P1

• Cline
• Goose
• Pi
• Copilot-CLI

Related

#1042
#1564