docs: training on rollouts from an external agent harness

[ananthsub]

Documents training on rollouts from an external agent harness that drives its own model calls.

Summary

• Explains run-level infrastructure enablement, static per-agent selection, all_agents, and the explicit /training-token-capture request path.
• Distinguishes training capture from neutral rollout correlation and evaluation observability.
• Documents async sink/source contracts, frozen snapshots, durable incomplete state, frozen retirement tombstones, and conditional retirement after downstream durability.
• Shows how a training framework supplies paired transport-backed endpoints without adding framework dependencies to Gym.
• Covers worker-local installation limits, startup validation, selected-agent finalization, reconstruction masking, rollout identity, and first-run metrics.
• Provides the caller-owned finalize_rollout_token_capture and retire_rollout_token_capture sequence.

The page is linked from the training tutorials index. Parent resolution and prefix supply are documented in #2349.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-ananthsub-tokidcap-docs.docs.buildwithfern.com/nemo/gym

Here are the markdown pages you've updated:

• nemo/gym/tutorials/training-tutorials/external-agent-harnesses
• nemo/gym/tutorials/training-tutorials

[ananthsub]

/claude review

[claude]

SHIP WITH CARE — docs-only PR (new external-agent-harnesses.mdx + one nav card). One inline finding.

Verified against the base nemo_gym/token_id_capture/ code and everything else in the doc is accurate: the config keys (enabled/dir/sink/sink_kwargs/rebuild_response, and source required when sink + rebuild_response=true), the _ng_token_capture metrics keys (n_calls, chains, delivered_fraction, quarantined_calls, empty_generation_calls, mask_sample), the finalize_rollout_token_capture / retire_rollout_token_capture signatures and their in-place / never-raise / retire-only-after-durable-handoff semantics, the _ng_rollout_id precedence and its ^[A-Za-z0-9][A-Za-z0-9._-]*$ validation, and the uvicorn num_workers > 1 sink-in-worker caveat all match the implementation.

The one issue (RISK, inline): the protocols code block at lines 100-108 is stale — it lists TokenSource.tokens_for/1-arg drop and a sync mark_incomplete, whereas the module defines freeze/drop(*, snapshot_id, version)/close and async mark_incomplete. It contradicts the doc's own downstream prose (freeze/snapshot/retire). Startup protocol-validation in config.py makes a literal implementation fail fast rather than silently, so this is a doc-accuracy fix, not a data-corruption risk. Details and a corrected block are in the inline comment.

[ananthsub]

/claude review

[ananthsub]

/claude review

[claude]

SHIP — LGTM, no reliability concerns.

Docs-only PR: a new training tutorial (external-agent-harnesses.mdx) plus an index card linking to it. Reviewed as documentation-accuracy against the trusted base, since a wrong API/config claim here would mislead people wiring up RL token capture.

Verified every material claim against nemo_gym/token_id_capture/:

• TokenSink / TokenSource protocol signatures (put, mark_incomplete, freeze, drop, close) match protocols.py exactly.
• finalize_rollout_token_capture / retire_rollout_token_capture from token_id_capture.delivery match delivery.py, including the freeze→handoff→retire ordering and mask_sample semantics.
• Config keys (enabled, all_agents, dir, sink, sink_kwargs, rebuild_response) match TokenIdCaptureSettings; extra="forbid" confirms the "typo raises an error" claim.
• TokenCaptureStore implements both protocols (default file store) — matches store.py.
• All _ng_token_capture metric keys (n_calls, chains, delivered_fraction, quarantined_calls, empty_generation_calls, top-level mask_sample) match consumer.py.
• Rollout-id rule (letters/digits/dots/dashes/underscores, must start with letter or digit; rejected not rewritten) matches rollout_correlation.maybe_rollout_id_from_run_body.
• sampling_overrides matches responses_api_models/vllm_model/app.py; per-agent token_id_capture: true flag matches base_responses_api_agent.
• Index-card href follows the existing /tutorials/training-tutorials/... convention.

No code paths touched; nothing to break at runtime.