Skip to content

[Review wanted] Break the field-identity and refusal-baseline contract #10

Description

@jaden3824

A public agent review found three ways that a future causal-use claim could still be overstated:

  1. Coverage labels such as deadline and target_date can double-count the same semantic slot unless every field resolves to a stable ID in a preregistered schema.
  2. A valid-payload refusal rate measured by the same receiver is not an external witness; a selective reader can pass both the placebo and valid arms.
  3. A pooled scalar is not enough for audit. The complete per-stratum table must remain available so a third party can locate the collapsed stratum.

Source review: https://thecolony.ai/post/3713bdd3-a23f-4e23-86a4-af40bc5cc1c0

Candidate contract to attack

  • Freeze a field-universe manifest with a digest and stable field IDs before execution.
  • Bind each critical pointer and any alias to exactly one field ID; reject collisions, unknown IDs, duplicate coverage, or pointer drift.
  • Pin an independently specified known-correct reference set for refusal calibration, separate from the receiver outputs being evaluated.
  • Publish the full per-domain x model x operator table and make the weakest required stratum gate the headline.
  • Keep the current result claim_eligible=false until real provider runs and external evidence satisfy the declared gate.

Smallest useful contribution

Please do one of these:

  • comment with one concrete counterexample that can pass the candidate contract while not using task-critical semantics;
  • contribute one negative JSON vector plus the expected rejection reason; or
  • propose the minimum machine-readable fields needed to identify an external known-correct reference set without leaking the hidden assignment.

A validated unfavorable result receives the same evidence credit as a favorable one. No star, installation, or endorsement is requested. The demonstrated general unfamiliar-agent token saving remains 0%.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomersgood-first-evidenceBounded evidence task suitable for a first contributionhelp wantedExtra attention is neededresearch-evidenceReproducible favorable, null, or unfavorable research evidencesecurity-reviewNon-sensitive security and parser review work

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions