Problem
The research-status-evidence-audit verifier returns nonzero aggregate reward for a submission whose evidence digest is invalid. The evidence is a mandatory digest-bound artifact; an invalid digest must make the aggregate reward zero while preserving the independent diagnostic.
Reproduction
Starting from the canonical COMPUTED submission, replace only the evidence digest with:
"evidence": [{"path": "evidence/answer.txt", "sha256": "sha256:0000000000000000000000000000000000000000000000000000000000000000"}]Running tests/verifier.py through run_verifier_in_child on revision 528500bf8691982d36d583dc7daf5f68e765cd32 returns:
{
"correctness": 1.0,
"evidence_validity": 0.0,
"scope_accuracy": 1.0,
"assurance_calibration": 1.0,
"reward": 0.2,
"false_certification": false
}tests/verifier.py:261-265 detects the failed digest binding, but tests/verifier.py:277-281 still adds the scope and assurance components after setting the evidence-dependent base to zero. The repository benchmark contract requires malformed or unbound evidence to receive zero aggregate reward; the independent evidence_validity=0.0 diagnostic should remain visible.
Problem
The
research-status-evidence-auditverifier returns nonzero aggregate reward for a submission whose evidence digest is invalid. The evidence is a mandatory digest-bound artifact; an invalid digest must make the aggregate reward zero while preserving the independent diagnostic.Reproduction
Starting from the canonical
COMPUTEDsubmission, replace only the evidence digest with:Running
tests/verifier.pythroughrun_verifier_in_childon revision528500bf8691982d36d583dc7daf5f68e765cd32returns:{ "correctness": 1.0, "evidence_validity": 0.0, "scope_accuracy": 1.0, "assurance_calibration": 1.0, "reward": 0.2, "false_certification": false }tests/verifier.py:261-265detects the failed digest binding, buttests/verifier.py:277-281still adds the scope and assurance components after setting the evidence-dependent base to zero. The repository benchmark contract requires malformed or unbound evidence to receive zero aggregate reward; the independentevidence_validity=0.0diagnostic should remain visible.