[MaxGhenis]

Flags the Colorado replay's weakly identified matches in the working paper and accounts for each case the replay does not reproduce. Publishes this as revision 11.

History. This PR was stacked on #95 (retire-qc-claims). #95 merged as 780f305 and #96 as c247b3d. This branch has been rebased onto main and retargeted to it. The original paper commits (1e2c45d, 773e170 and a945cde) are now 9fd731b, cf7b1aa and f070578. Four commits follow them:

• d7e0c08 carries Correct the amterr lab's solver description against the R code #96's final counts into the paper, FACTS D5 and the tests;
• fea4a81 publishes the change as revision 11;
• 7c074f6 rewords the two sentences that Correct the amterr lab's solver description against the R code #96's round-4 review flagged;
• d3460a2 addresses this PR's round-3 review nits: D5's push sentence, and a lock on the revision label.

What it changes in the paper

Replay paragraph (#sec-decompose)

• Weak identification, after the threshold rates: "In 64 of the 230 moved matches, 32 of them above the threshold, the issued benefit lies within $5 of a stretch where the solver's benefit formula is flat in the moved input (the maximum allotment in 53, the minimum benefit in 6, rent past the shelter-deduction cap in 3 and rent low enough for a zero shelter deduction in 2). By that formula every amount of the input on the stretch reproduces the issued benefit, so the match bounds the input on one side at most and is weakly identified. In 34 of the 53 the issued benefit is the maximum allotment and the solver lowered an income; the solver pays the maximum whenever net income is zero or less, so every value of that income up to the point where net income reaches zero yields it."
  • This is the caveat that round 3 of Correct retired SNAP QC claims in the paper, README, FACTS and simulator #95's review called "the natural caveat on the 78.4% and 91.4%" (R8).
  • The counts are Correct the amterr lab's solver description against the R code #96's final ones (claims_audit.json, case_level.solver_outcomes.weakly_identified). Correct the amterr lab's solver description against the R code #96 pushes the moved input both ways from the solver's amount and adds the zero-shelter stretch, so 60 = 53 + 4 + 3 became 64 = 53 + 6 + 3 + 2, and the above-threshold count went from 31 to 32.
• Accounting of the 37 misses. The solver's-limits sentence now covers each one exactly once:
  • 17 where nothing moved, which the paper already said;
  • "In 6 more, its $3 steps reached within $3 of the issued benefit before the utility reset moved the input off that match";
  • 9 where a stop rule ended the steps short;
  • 5 household-size misses.
• "Of the 10, 2 are among the 6 the utility reset moved off a match."
• "a stepped utility amount is then reset" now reads "the utility amount is then reset". Two rows were reset without a step.

Footnote and FACTS D4. The stop rules now say "at or above the maximum allotment" (202311-40369 has RAWBEN above it). They also add the mirror rule: at or below a one- or two-person unit's minimum benefit, income-raising steps run until the uncapped benefit falls below zero.

New FACTS rows

• D5: weak identification. It covers the both-ways push, the 61 / 3 direction split and its keys, and the counts by stretch (53, 6, 3, 2). It also gives the 3 matches bounded on neither side, the 2 household-size matches that are not assessed, the 34 income cuts at the maximum, the 5 matches not counted, and the at_max flag's 54. The wording matches paper/snapshot/labs/amterr/ANALYSIS.md with 7c074f6's two rewordings.
  • D5 states the push test as a condition: "The benefit is monotone in each input, so when a pushed amount still gives a benefit within $5 of RAWBEN, every amount between the solver's and that one does too; a match counts when that pushed amount also lies on one of the flat stretches named below." The $100,000 is quoted from audit_claims.SOLVER_PUSH.
• D6: what the solver did in the 20 moved misses, the 3 moves that end farther from RAWBEN than FSBEN, and the audit's two checks of D4.

Lab write-up. ANALYSIS.md picks up the same two rewordings as D5 (7c074f6):

• "so they too bound the input on one side only, from above";
• "the solver moves household size once, by one person, so there is no range of amounts to scan".

Abstract and headline claims

The abstract is unchanged. It says the engine "reproduces the issued benefit within $5 for 78.4% … and 91.4% … which is consistent with correct arithmetic on a wrong input". That claim still holds for every weakly identified match. The caveat qualifies how much those matches show; it doesn't change a headline number.

Rendering and revision

• Re-rendered at the final head with Quarto 1.9.36 (cd paper && quarto render index.qmd). paper/out/index.html is byte-identical to the committed app/public/paper/web/index.html. The PDF's text is identical (pdftotext); only its creation timestamps and document ID differ.
• Revision 11 (2026-10-24). Revision 10 is live: it was deployed for d934. So this publishes a new revision:
  • the date moves in index.qmd;
  • the wrapper (app/public/paper/index.html) gets a new description, the tag "Revision 11 · 2026-10-24" and the cache key r11-20261024.

Tests

• tests/test_retired_claims.py:
  • replay_quotes locks the paper's sentences against claims_audit.json;
  • the Hypothesis count strategy draws the four stretch counts;
  • test_replay_counts_partition checks 53 + 6 + 3 + 2 = 64, 34 ≤ 53, and 14 + 3 + 6 + 9 + 5 = 37;
  • test_facts_quotes_the_solver_outcomes locks D5 and D6, including the push sentence, the direction split, the neither-side keys, the household-size clause and the "Not counted" sentence.
• tests/test_paper_embed.py: test_revision_label_matches_the_manuscript_date checks that the wrapper's tag, its description and the cache key name one revision, dated as paper/index.qmd.
• Mutations: six edits each failed a lock. They changed the tag's number, the tag's date, the description's revision, the manuscript date, D5's $100,000, and D5's "also lies on".
• audit_claims.py --check reports "is current".
• tests/test_retired_claims.py, tests/test_amterr_lab.py, tests/test_cause_shares.py and tests/test_paper_embed.py give 71 passed at 7c074f6, none skipped, with the QC postings present.

Review

~/reviews/snap-qc-sim-pr97/review-r3.md (in-session Opus; every Subfleet lane was closed) approved 7c074f6 with three nits. d3460a2 addresses nits 1 and 3. From nit 2 it takes the revision-label lock; the footnote's stop rules stay unlocked, as round 1 accepted.

axiom: n/a: manuscript text and fact catalog; no policy encoding changes

🤖 Generated with Claude Code