Problem
mathematical-benchmarks-v1/graph-counterexample is documented as a self-contained offline task, but its agent environment declares network_mode = "public". Harbor resolves that mode to unrestricted network access, so an agent trial can make arbitrary outbound requests even though the task has no external-data requirement and explicitly says web search is disabled.
Evidence
The task contract declares the agent environment as public at benchmarks/datasets/mathematical-benchmarks-v1/graph-counterexample/task.toml:35-39. The same task says the input is offline and self-contained at instruction.md:3 and README.md:27-31, and its metadata tags the task offline at task.toml:22.
Harbor's Docker backend applies public as allow-all (harbor/environments/docker/docker.py:_apply_network_policy), while the job's extra_allowed_hosts list is ignored for a public policy (harbor/trial/network_policy.py:28-35). Thus the committed job configuration does not narrow the task's egress.
Reproduction
- Load
task.toml for graph-counterexample.
- Resolve the task's agent environment policy: it is
public.
- Run the task under the Docker environment. Harbor sends the egress controller the
allow-all policy for the agent phase.
- The agent can reach arbitrary external hosts, despite the task's offline/self-contained contract.
The separate verifier environment is correctly no-network at task.toml:41-45; the issue concerns the agent phase.
Expected behavior
The agent environment should use no-network for this self-contained task, or an explicit allowlist if a documented runtime dependency is required. The task metadata, README, instruction, and job policy should agree so that an offline task cannot silently run with unrestricted egress.
Problem
mathematical-benchmarks-v1/graph-counterexampleis documented as a self-contained offline task, but its agent environment declaresnetwork_mode = "public". Harbor resolves that mode to unrestricted network access, so an agent trial can make arbitrary outbound requests even though the task has no external-data requirement and explicitly says web search is disabled.Evidence
The task contract declares the agent environment as public at
benchmarks/datasets/mathematical-benchmarks-v1/graph-counterexample/task.toml:35-39. The same task says the input is offline and self-contained atinstruction.md:3andREADME.md:27-31, and its metadata tags the taskofflineattask.toml:22.Harbor's Docker backend applies
publicasallow-all(harbor/environments/docker/docker.py:_apply_network_policy), while the job'sextra_allowed_hostslist is ignored for a public policy (harbor/trial/network_policy.py:28-35). Thus the committed job configuration does not narrow the task's egress.Reproduction
task.tomlforgraph-counterexample.public.allow-allpolicy for the agent phase.The separate verifier environment is correctly
no-networkattask.toml:41-45; the issue concerns the agent phase.Expected behavior
The agent environment should use
no-networkfor this self-contained task, or an explicit allowlist if a documented runtime dependency is required. The task metadata, README, instruction, and job policy should agree so that an offline task cannot silently run with unrestricted egress.