You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two scorer issues surfaced during the #1012 Phase-3 A/B (both worked around, neither blocking):
Caller token shadows the fallback (fix(qa): scorer keychain-token fallback (#1260 round 3 — final) #1266 gap):qa/score.sh's keychain-derivation only fires when CLAUDE_CODE_OAUTH_TOKEN/ANTHROPIC_API_KEY are absent — a STALE caller-provided token (e.g. exported once for the DM's auth at run start, expired by scoring time) shadows the fix and reproduces 401s. Fix: on a 401 from a caller-provided token, retry once with a freshly-derived keychain token before declaring scorer_failed.
Retry-without-truncate artifact: failed attempts leave trailing JSON fragments after the first valid object in the OUT file (observed: ab1012b-*.score.json). First-object parse succeeds so consumers work, but the files aren't clean JSON. Fix: truncate/rewrite OUT atomically per attempt.
Two scorer issues surfaced during the #1012 Phase-3 A/B (both worked around, neither blocking):
qa/score.sh's keychain-derivation only fires whenCLAUDE_CODE_OAUTH_TOKEN/ANTHROPIC_API_KEYare absent — a STALE caller-provided token (e.g. exported once for the DM's auth at run start, expired by scoring time) shadows the fix and reproduces 401s. Fix: on a 401 from a caller-provided token, retry once with a freshly-derived keychain token before declaring scorer_failed.ab1012b-*.score.json). First-object parse succeeds so consumers work, but the files aren't clean JSON. Fix: truncate/rewrite OUT atomically per attempt.