fix(judge): a judge outage is unjudged, not an agent failure

[vaibhavdabas16]

What does this PR do?

Fixes #299 — all three asks.

judge.py:266 returns match=None when the judge call fails after retries: HTTP 402, a timeout, an unsupported api_type. run.py exited 1 for that, and batch.py counts exit 1 as failed. So a judge account hitting 402 mid-sweep recorded whole batches as agent failures and silently deflated reward — the case the issue reports as already having happened.

The run itself was complete. It intercepted; only stage 2 was missing.

Ask 1 — a distinct outcome

run.py already printed JUDGE MISMATCH and JUDGE INCONCLUSIVE differently on that branch, then threw the distinction away by exiting 1 for both. It now exits JUDGE_INCONCLUSIVE_EXIT (3) when the verdict is None, and keeps 1 for a genuine mismatch — where the judge did answer, and the answer was no.

Ask 2 — counted separately

batch.py maps that code to a new unjudged status, so batch-summary.json shows "N runs unjudged" instead of burying them in failed.

The status vocabulary was spelled out twice — once in print_summary, once in write_summary_json — which is how a new status would have been added to one and missed in the other. Both now read a single JOB_STATUSES tuple.

Ask 3 — targeting them for re-judging

clawbench-rescore --only-unjudged re-scores exactly those runs, which is the cheap recovery path once a provider comes back rather than re-judging a whole sweep.

The predicate is shared: is_judge_inconclusive() lives in run_support/results.py beside classify_run, so the runner, the batch driver and the scorer cannot drift on what "unjudged" means. A --no-judge run has no judge_match key and is deliberately not inconclusive — nothing was attempted, so nothing is outstanding; without that, the entire --no-judge corpus would look like it needed re-judging.

The --resume half of ask 3 is not here — see the note on #329 below.

Two things worth a maintainer's eye

1. This changes an assertion from #314. test_run_judge_stage.py::test_judge_failure_after_the_run_still_writes_run_meta asserted excinfo.value.code == 1 with the comment "Inconclusive judge -> normal exit 1". That was #314 pinning the exact behaviour #299 calls a bug, so the assertion moves to JUDGE_INCONCLUSIVE_EXIT. What the test actually guards — a clean exit rather than an uncaught crash, and run-meta.json written for a completed run — is unchanged.

2. Semantic overlap with #329, which git does not flag. git merge-tree says this branch merges cleanly with #329 (fix/batch-resume-from-run-meta), but #329's recorded_outcome() maps a run to a batch status and checks intercepted first. A judge-outage run is intercepted, so after both land a resumed batch would carry it over as passed — worse than the failed this PR is fixing. The fix is one branch in recorded_outcome: check is_judge_inconclusive(meta) before intercepted and return "unjudged". I'll rebase whichever of the two lands second and add exactly that; flagging it now because a clean merge will not surface it.

Corpus

• v2
• v1
• both
• not applicable

_{Host-side scoring/reporting; no task data involved. No published number changes — this separates outcomes that were previously conflated, it does not re-grade anything.}

Test plan

• New tests/test_judge_inconclusive.py, 14 tests: what counts as inconclusive (outage vs match vs mismatch vs --no-judge vs never-intercepted); the exit code being distinct from 0/1/2; run.py using it only when the verdict is missing; unjudged being its own status; batch-summary.json counting it apart from failed; the run.py↔batch.py wiring; and --only-unjudged selecting exactly the outage runs.
• Control: restored sys.exit(1) and dropped the unjudged status with the tests held constant — 4 of the 14 fail, including the summary-totals one.
• rescore must not start requiring Docker: it now imports run_support.results, which has no import-time engine probe. Verified by importing clawbench.eval.rescore in a subprocess with an emptied PATH, so the probe would fire if it were reachable.
• An unreadable run-meta.json is skipped by --only-unjudged rather than crashing the scan.
• Full suite: 224 passed, 4 skipped. The 4 deselected are the Windows-only git-symlink case, green on CI.
• ruff format --check clean across src, tests, scripts (87 files). ruff check under the ruleset CI resolves (E4,E7,E9,F): all checks passed. pyright --pythonplatform Linux src/clawbench tests: 0 errors.
• Merge-clean against main and all six of my other open branches (with the fix(batch): resume from run-meta.json, not from log-file existence #329 caveat above, which is semantic rather than textual).

Not verified: no live run — I don't have Docker on this machine, so the 402-mid-sweep path is exercised by injection rather than against a real judge outage.

Related issues

Fixes #299. Related: #243 asks for the same separation between stage 1 and stage 2 in the reported numbers; this makes the stage-2 gap visible, which is the half #243 needs.

_{Exit codes were undocumented, which a third one makes worse — docs/cli.md now lists all four, plus the new --only-unjudged flag.}