Skip to content

feat: group stacked redaction leaks into one recoverable block - #2

Merged
jonwiggins merged 1 commit into
mainfrom
jonwiggins/merge-adjacent-redaction-blocks
Jul 20, 2026
Merged

feat: group stacked redaction leaks into one recoverable block#2
jonwiggins merged 1 commit into
mainfrom
jonwiggins/merge-adjacent-redaction-blocks

Conversation

@jonwiggins

Copy link
Copy Markdown
Member

Description

Redaction tools draw one black box per line, so a poorly-redacted paragraph surfaced in the results view as a dozen separate cards with its recovered text split across all of them. checkPage now merges leak findings that stack into a single block — same check, horizontally-overlapping columns, and a vertical gap up to ~1.25 line-heights (so double-spaced filings like the Manafort response merge correctly, where consecutive redaction boxes sit a full line-height apart). The merged finding spans the whole block (so the "show in document" overlay frames the passage) and carries the recovered text joined top-to-bottom, left-to-right.

Result on the Manafort fixture: page 5 goes from 9 fragment cards to 2 readable paragraphs, and the recovered text reads back in full.

This is the only change that affects the shipped bundle (src/lib/analyzer/checks.ts); the rest is tests, fixtures, and docs.

Regression set updates

  • Add the 2005 Calipari / Sgrena US Army AR 15-6 report — a textbook black-box-over-live-text failure (grade F, hundreds of recovered names/details). Verified the recovered text sits under solid black fills via a render, and that the pinned source URL reproduces the SHA-256. PDF stays gitignored per the existing real-world convention.
  • Reclassify the TSA SOP fixture from failure to control. The fetchable cryptome copy is a reveal edition: the once-hidden passages are printed openly with a red outline around each (a clip path + red stroke, no dark fill). Nothing is actually covered, so grade A is correct — the old F was a false positive. It now guards exactly that case.
  • Rebaseline Manafort/JPMorgan minLeaks for the per-passage merge.

Also

  • New tests/merge-findings.test.ts (7 cases pinning the merge geometry: double-spacing, indents, paragraph-break gaps, side-by-side columns, cross-check isolation).
  • .gitignore now covers scans/, election-integrity/, and dni-scan/ so a stray git add -A can't commit a downloaded release.

Testing

  • npm test → 24 pass (12 synthetic examples, 7 merge-geometry, 5 real-world incl. the new Calipari fixture).
  • npm run lint clean; npm run build succeeds.
  • Real-world PDFs are fetched on demand (npm run examples:real) and skip in CI when absent, so the suite stays green without them.

Redaction tools draw one black box per line, so a poorly-redacted
paragraph surfaced as a dozen separate findings with its recovered text
split across all of them. checkPage now merges leak findings that stack
into a single block (same check, overlapping columns, vertical gap up to
~1.25 line-heights so double-spaced filings like the Manafort response
merge correctly). The merged finding spans the whole block and carries
the recovered text joined in reading order, so a hidden passage reads
back in full and the overlay frames the whole thing.

Also updates the real-world regression set:
- add the 2005 Calipari/Sgrena US Army report (a textbook black-box-over-
  live-text failure; grade F, hundreds of recovered names/details);
- reclassify the TSA SOP fixture from failure to control — the fetchable
  cryptome copy is a reveal edition (once-hidden text shown openly with a
  red outline, nothing actually covered), so grade A is correct and it now
  guards that exact false positive;
- rebaseline Manafort/JPMorgan leak counts for the per-passage merge.

Adds tests/merge-findings.test.ts and gitignores scan working dirs and
downloaded releases so they can never be committed.
@jonwiggins
jonwiggins merged commit 5f0e451 into main Jul 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant