feat(agents): RemoteAgent, thin proxy server for user-hosted remote agent services

[adil-a]

Part of the external-agent-integration epic #1396. Supersedes the collector-side agent_url approach proposed in #2006 (closed unmerged in favor of this PR).

What

An agent server, responses_api_agents/remote_agent/, that drives an agent service running outside Gym's process tree (your own repo, your own infrastructure) through Gym's tool loop. The remote service implements one endpoint compliant with the OpenAI /v1/responses contract, and the two servers compose as Responses-speaking agents:

POST {agent_base_url}/v1/responses — receives the conversation so far (the row's responses_create_params with accumulated output and tool results appended to input) and returns a Responses API object: unpaired function_call items to ask Gym to execute environment tools, paired call+output items as records of its own internal tools (passed through untouched), or a final assistant message to finish.

Gym owns everything else: it seeds the session and holds its cookies, executes the asked-for tools against the resources server, appends results and calls the service again, validates each reply against the Responses API schema (failing the rollout on mismatch), verifies, and returns the verify response from /run. The resources server is never exposed to the service — tool execution, session cookies, and verifier_metadata all stay inside Gym. The service's own cookies are round-tripped per call so it can keep per-rollout state; the only network direction is Gym → service. To the collector it is a normal named agent — resume, aggregation, and profiling work unchanged.

The loop is simple_agent's responses() with two marked in-loop divergences — the model hop is a hardened POST to agent_base_url, and only unpaired function_calls are executed (a call the service already answered itself is its own record, not an ask) — plus the never-raise /run contract that turns transport/validation errors into failure rows, and one deliberate cookie difference: the service's own cookies are round-tripped to it but stay out of the outgoing Set-Cookie (they are its private session, not Gym's). run() mirrors simple_agent's seed → self-post /v1/responses → verify.

Why a server instead of collector-side agent_url (#2006)

• The token-capture/training stack (feat(token-id-capture): capture training tokens from external harnesses #2124–feat(token-id-capture): Claude Code external-harness example #2128, unmerged) gates participation by the agent's name in Gym's config; a url-dispatched agent has no config entry and is excluded by construction. A named RemoteAgent is compatible by design — note this PR contains no capture wiring; a wiring example is a follow-up once that stack lands.
• Failure handling, validation, timeouts, and session plumbing live in the agent-server layer — this server's /run implementation plus the shared SimpleServer/ServerClient plumbing — instead of being reimplemented inside the shared collection loop.
• /run responsibilities stay in the agent-server layer; the collector stays agent-agnostic.
• Same pattern as every other harness integration (claude_code_agent, codex, …).

Failure contract

Failures never raise out of /run: remote endpoint down (3 total connection attempts, ClientOSError/ServerDisconnectedError only; timeouts and all other errors fail after a single attempt), per-call and whole-rollout timeouts, malformed/interrupted replies, seed/verify errors, and internal bugs all become reward-0 sentinel verify-responses (_ng_failure_class="remote_agent_error") that rollout collection routes to the failures sidecar and retries on resume — non-terminal failures only, up to NEMO_GYM_MAX_ROLLOUT_ATTEMPTS (default 3). An invalid Responses object from the service is terminal (a schema bug will not fix itself on retry); the terminal flag crosses the HTTP self-post boundary by exception name. Reused rollout/failures JSONL rows carrying stale result keys are sanitized rather than crashing or leaking routing flags. Tool-level errors are NOT rollout failures: an unknown tool name or malformed arguments come back to the service as that call's function_call_output, matching simple_agent's semantics.

Testing

• 51 offline tests (mocked ServerClient + mocked aiohttp seam; the /v1/responses self-post is routed into the real responses() with the exception middleware emulated): config validation, loop mechanics (multi-turn tool execution, paired-call pass-through, unknown-tool feedback, malformed-arguments feedback, max_steps, service-cookie round-trip, usage accumulation), every transport failure mode → sentinel, terminal classification across the route boundary, semaphore bounds incl. release-on-failure, run-wallclock-after-semaphore semantics, aggregate proxy + bound, route-level serialization (HTTP 200, never 500).
• Stateful E2E in-suite: the real example_session_state_mgmt counter server in-process; the service returns unpaired tool asks, GYM executes them on the seeded session, reward 1.0 through the real verifier — and the test asserts the service was fed the counter value Gym read back.
• Collector round-trip in-suite: real RolloutCollectionHelper.run_from_config driving this agent — successes to the main JSONL, sentinel rows to the failures sidecar.
• Live E2E, off-host, real agent: a containerized service (own network namespace) wrapping the Claude CLI (opus via an internal gateway) as the decision-maker, agent_base_url pointing at the container's bridge IP; the counter resources server stayed on loopback with a random port — unreachable from the container by construction. 5/5 rollouts reward 1.0, trajectories reading function_call → function_call_output → … → message, real token usage accumulated across loop turns (~8.3k mean/rollout). The only network path was Gym → container.

Follow-ups

• run-mode (/run-exposing self-scoring services) as a config mode on this server
• token-capture wiring example once feat(token-id-capture): capture training tokens from external harnesses #2124–feat(token-id-capture): Claude Code external-harness example #2128 land (the service would route its model calls through Gym's model server at the per-rollout prefixed URL)
• per review discussion: opt-in bounded-retry/timeout kwargs on core request() (then this server's transport collapses onto core), and two additive core helpers with existing adopters — a failure-row builder (stirrup/pinchbench hand-roll the same) and an aggregate-metrics proxy helper (eight agents hand-roll it)

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-remote-agent.docs.buildwithfern.com/nemo/gym

Here are the markdown pages you've updated:

• nemo/gym/agent-server/agent-skills
• nemo/gym/agent-server/remote-agent

[ffrujeri]

The remote service implements one endpoint and is called like a model:

I think here we don't necessarily need to say that it is called like a model, because it is more like a composition of agents all compliant with the OpenAI Responses contract. And with the possibility of calling Gym's resources servers tools