chore(hermes_agent): evaluate replacing the fork commit with a named upstream release

[cwing-nvidia]

Summary

Evaluate moving responses_api_agents/hermes_agent from its current commit-pinned fork to an exact named upstream Hermes Agent release. Keep the dependency reproducible; this is not a proposal to install latest without validation.

Current state

Gym currently installs:

hermes-agent @ git+https://github.com/cmunley1/hermes-agent@26bb847a

That fork commit is dated May 7, 2026. The Hermes README says the fork carries patches for token-ID tracking, chat-template behavior, and sampling parameters needed for training. The in-process Gym adapter also depends directly on the fork's run_agent.AIAgent API.

Upstream Hermes has continued evolving, including changing native MCP tool names from the fork's mcp_<server>_<tool> form to the mcp__<server>__<tool> convention Gym already normalizes.

Why reconsider the fork

A named release would give the integration a clearer compatibility and upgrade contract than an aging commit on a personal fork. More importantly, Gym's newer model-server token-capture work may allow external agent harnesses to participate in training without maintaining every training-specific patch inside each harness fork.

Relevant work from @ananthsub includes:

• feat(vllm-model): pin sampling params for on-policy training #2190 — merged model-server sampling overrides for on-policy training
• feat(token-id-capture): deliver rebuilt trajectories safely #2126 — merged durable token capture and rebuilt trajectory delivery
• feat(token-id-capture): resolve each call's parent at request time #2180 — open request-time parent resolution for multi-call trajectories
• docs: training on rollouts from an external agent harness #2341 — open guide for training on rollouts from an external agent harness
• docs: parent resolution and verified prefix supply #2349 — open token-capture data-plane and prefix-supply documentation/tests

These changes do not automatically prove the Hermes fork is unnecessary. They make it worth re-evaluating which fork patches remain required and which responsibilities can now be provided generically by Gym's model server.

Investigation

• Inventory every behavior Gym relies on from cmunley1/hermes-agent@26bb847a, including token IDs/logprobs, sampling forwarding, chat templates, reasoning/history preservation, and the AIAgent API.
• Identify the first suitable named upstream Hermes release and compare it with the fork.
• Determine which training requirements are satisfied by Gym's token-capture and sampling infrastructure versus which still require harness changes.
• Check whether upgrading removes the Hermes-specific MCP alias incompatibility described in Preserve structured MCP tool identity for verification #2668.
• Assess migration work required in hermes_agent/app.py, tests, container dependency installation, and documentation.

Acceptance criteria

• The fork delta and every Gym dependency on it are documented.
• A specific named Hermes release is selected and pinned exactly, or concrete blockers to using a named release are documented.
• If migration is viable, the personal-fork commit dependency is removed.
• Existing Hermes unit tests pass, including token metadata and sampling behavior.
• Training compatibility is validated against the external-harness token-capture contract rather than assumed from unit tests.
• A real multi-turn Hermes rollout is inspected for trajectory fidelity, MCP naming, token capture, and reward behavior.
• The README accurately describes the final dependency and training contract.

[cwing-nvidia]

Latest Hermes agent v0.20.5 explicitly refuses wheel/sdist installation and is supported only through its managed CLI installer, Docker, or Nix. Gym’s current Hermes adapter imports AIAgent in-process, so v0.20.5 is not a simple dependency bump