Skip to content

Add preprocess-legal plugin (slice 1: text-native extraction + confidence gate) - #105

Open
emtcmca wants to merge 1 commit into
anthropics:mainfrom
emtcmca:add-preprocess-legal-plugin
Open

Add preprocess-legal plugin (slice 1: text-native extraction + confidence gate)#105
emtcmca wants to merge 1 commit into
anthropics:mainfrom
emtcmca:add-preprocess-legal-plugin

Conversation

@emtcmca

@emtcmca emtcmca commented Aug 1, 2026

Copy link
Copy Markdown

Addresses #100 (local pre-processing before large local file sets enter context) with a gate, not just an extractor.

What this does

A PreToolUse hook fires on each Read. For a routed PDF the engine extracts text-native content locally, scores per-document extraction confidence, and rewrites the read's file_path (updatedInput) to point at either clean text on a pass or an honest withheld-marker on a fail. Because the engine writes the exact bytes the model reads, "refuse to pass degraded output silently" is literal rather than advisory.

Why the gate, and not just the extraction

A text extractor run against a scanned exhibit returns something, and the something often looks like text. Passing that downstream trades a token cost for a silent correctness cost, which in a case file is the expensive direction to be wrong in. It is also why auto-trigger is the setting everyone would leave off: nobody wants a fast path that might quietly hand them a hollowed-out exhibit.

Confidence is scored as an explainable breakdown rather than a single opaque number:

  • coverage — fraction of pages that produced real text
  • density — mean characters per page against a text-native baseline
  • legibility — fraction of characters that are actually readable
  • overall = (weighted coverage + density) × legibility

Legibility acts as a quality veto, so a page of plausible-looking mojibake cannot pass on quantity alone.

Two corrections to the design sketched in the issue thread

Both were established by testing the hook mechanism rather than reading the docs:

  1. There is no folder-level ingestion event. Hooks fire per tool call, so interception is inherently per-file and routing is per document. The "is this folder expensive" heuristic in my original comment is not load-bearing.
  2. PostToolUse cannot replace what the model sees. It is additive, and block + reason shows the model the original plus the reason. PreToolUse redirect via updatedInput.file_path is the mechanism that works, and it also saves the raw file IO rather than only the token cost.
Mechanism Result
PostToolUse rewriting tool output model still saw the original
PostToolUse block + reason model saw the original plus the reason
PreToolUse redirect (updatedInput.file_path) model transparently saw the substituted content

Scope and limits

  • Opt-in. Disabled unless a config sets enabled: true.
  • Fails safe. A missing or broken config, a missing pdfplumber, or an encrypted or corrupt PDF never breaks a Read.
  • Swappable backend. Only engine/extract.py imports pdfplumber; scoring and gating operate on a plain ExtractionResult, so the deferred OCR cascade plugs in behind the same interface without touching the gate.
  • Slice 1 is text-native only. OCR fallback for low-confidence scans is deferred to slice 2.

Testing

15 tests covering the scoring and gate logic. They require no PDF backend:

python -m pytest external_plugins/preprocess-legal/tests/

scripts/lint-tool-scope.py passes clean. The plugin lives entirely under external_plugins/preprocess-legal/ and touches nothing else.

Open question for maintainers

Carried over from the issue thread and still open: does a hook that intercepts document reads belong in a practice-area plugin, or closer to Cowork itself? It is built behind a clean interface so the trigger is swappable either way, and I am happy to move it if this is the wrong home.

…ence gate)

Addresses issue anthropics#100 (local pre-processing before large local file sets enter
context) with a gate, not just an extractor.

A PreToolUse hook fires on each Read. For a routed PDF the engine extracts
text-native content locally, scores per-document extraction confidence, and
rewrites the read's file_path (updatedInput) to point at either clean text on a
pass or an honest withheld-marker on a fail. Because the engine writes the exact
bytes the model reads, "refuse to pass degraded output silently" is literal
rather than advisory.

Why the gate matters: a text extractor run against a scanned exhibit returns
something, and the something often looks like text. Passing that downstream
trades a token cost for a silent correctness cost, which in a case file is the
expensive direction to be wrong in. Confidence is scored as an explainable
breakdown (coverage, density, legibility) where legibility acts as a quality
veto, so a page of plausible-looking mojibake cannot pass on quantity alone.

Two corrections to the design sketched in the issue thread, both established by
testing the hook mechanism rather than reading the docs:

- There is no folder-level ingestion event. Hooks fire per tool call, so
  interception is inherently per-file and routing is per document.
- PostToolUse cannot replace what the model sees (it is additive, and block +
  reason shows the original plus the reason). PreToolUse redirect via
  updatedInput.file_path is the mechanism that works, and it also saves the raw
  file IO rather than only the token cost.

Scope and limits:

- Opt-in. Disabled unless a config sets enabled: true.
- Fails safe. Missing or broken config, missing pdfplumber, or an encrypted or
  corrupt PDF never breaks a Read.
- Swappable backend. Only engine/extract.py imports pdfplumber; scoring and
  gating operate on a plain ExtractionResult, so the deferred OCR cascade plugs
  in behind the same interface without touching the gate.
- Slice 1 is text-native only. OCR fallback for low-confidence scans is
  deferred to slice 2.

15 tests cover the scoring and gate logic and require no PDF backend.

Open question for maintainers, carried over from the issue thread: does a hook
that intercepts document reads belong in a practice-area plugin, or closer to
Cowork itself? Built behind a clean interface so the trigger is swappable either
way.
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant