nemo gym resources server required

[cmunley1]

Describe the bug

Environments in nemo gym do not all use a resources server. Without looking into full detail of the integration, this will exclude many environments from being runnable here.

nemo-platform/docs/evaluator/agent-eval/gym-runner.mdx

Line 125 in 5ea8be4

| `resources_server` | yes | resources-server (environment) name, e.g. `mcqa` |

A clear and concise description of what the bug is.

Steps/Code to reproduce bug

Please list minimal steps or code snippet for us to be able to reproduce the bug.

A helpful guide on on how to craft a minimal bug report http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

[Mahnoor-Zaffar]

Hey! Thanks for reporting this.

The most likely root cause is that the configuration for the environments does not consistently specify the use of a resources server. The relevant file for this configuration appears to be .mcp.json, which defines the mcpServers settings. Specifically, the nemo server is set to use a command that runs the nemo-mcp service, but it may not be correctly integrated across all environments in the NeMo Gym.

To address this issue, it would be reasonable to review the environment configurations to ensure that they all reference the resources server correctly. This may involve updating the environment setup to include the necessary server configurations.

If additional details about specific environments that are failing to run are available, that information would be helpful for further investigation.

If no one is currently working on this, I'd be happy to take a look and put together a fix.