Add preprocess-legal plugin (slice 1: text-native extraction + confidence gate) - #105
Open
emtcmca wants to merge 1 commit into
Open
Add preprocess-legal plugin (slice 1: text-native extraction + confidence gate)#105emtcmca wants to merge 1 commit into
emtcmca wants to merge 1 commit into
Conversation
…ence gate) Addresses issue anthropics#100 (local pre-processing before large local file sets enter context) with a gate, not just an extractor. A PreToolUse hook fires on each Read. For a routed PDF the engine extracts text-native content locally, scores per-document extraction confidence, and rewrites the read's file_path (updatedInput) to point at either clean text on a pass or an honest withheld-marker on a fail. Because the engine writes the exact bytes the model reads, "refuse to pass degraded output silently" is literal rather than advisory. Why the gate matters: a text extractor run against a scanned exhibit returns something, and the something often looks like text. Passing that downstream trades a token cost for a silent correctness cost, which in a case file is the expensive direction to be wrong in. Confidence is scored as an explainable breakdown (coverage, density, legibility) where legibility acts as a quality veto, so a page of plausible-looking mojibake cannot pass on quantity alone. Two corrections to the design sketched in the issue thread, both established by testing the hook mechanism rather than reading the docs: - There is no folder-level ingestion event. Hooks fire per tool call, so interception is inherently per-file and routing is per document. - PostToolUse cannot replace what the model sees (it is additive, and block + reason shows the original plus the reason). PreToolUse redirect via updatedInput.file_path is the mechanism that works, and it also saves the raw file IO rather than only the token cost. Scope and limits: - Opt-in. Disabled unless a config sets enabled: true. - Fails safe. Missing or broken config, missing pdfplumber, or an encrypted or corrupt PDF never breaks a Read. - Swappable backend. Only engine/extract.py imports pdfplumber; scoring and gating operate on a plain ExtractionResult, so the deferred OCR cascade plugs in behind the same interface without touching the gate. - Slice 1 is text-native only. OCR fallback for low-confidence scans is deferred to slice 2. 15 tests cover the scoring and gate logic and require no PDF backend. Open question for maintainers, carried over from the issue thread: does a hook that intercepts document reads belong in a practice-area plugin, or closer to Cowork itself? Built behind a clean interface so the trigger is swappable either way.
|
All contributors have signed the CLA ✍️ ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #100 (local pre-processing before large local file sets enter context) with a gate, not just an extractor.
What this does
A
PreToolUsehook fires on eachRead. For a routed PDF the engine extracts text-native content locally, scores per-document extraction confidence, and rewrites the read'sfile_path(updatedInput) to point at either clean text on a pass or an honest withheld-marker on a fail. Because the engine writes the exact bytes the model reads, "refuse to pass degraded output silently" is literal rather than advisory.Why the gate, and not just the extraction
A text extractor run against a scanned exhibit returns something, and the something often looks like text. Passing that downstream trades a token cost for a silent correctness cost, which in a case file is the expensive direction to be wrong in. It is also why auto-trigger is the setting everyone would leave off: nobody wants a fast path that might quietly hand them a hollowed-out exhibit.
Confidence is scored as an explainable breakdown rather than a single opaque number:
coverage— fraction of pages that produced real textdensity— mean characters per page against a text-native baselinelegibility— fraction of characters that are actually readableoverall = (weighted coverage + density) × legibilityLegibility acts as a quality veto, so a page of plausible-looking mojibake cannot pass on quantity alone.
Two corrections to the design sketched in the issue thread
Both were established by testing the hook mechanism rather than reading the docs:
PostToolUsecannot replace what the model sees. It is additive, andblock+ reason shows the model the original plus the reason.PreToolUseredirect viaupdatedInput.file_pathis the mechanism that works, and it also saves the raw file IO rather than only the token cost.block+ reasonupdatedInput.file_path)Scope and limits
enabled: true.pdfplumber, or an encrypted or corrupt PDF never breaks aRead.engine/extract.pyimportspdfplumber; scoring and gating operate on a plainExtractionResult, so the deferred OCR cascade plugs in behind the same interface without touching the gate.Testing
15 tests covering the scoring and gate logic. They require no PDF backend:
scripts/lint-tool-scope.pypasses clean. The plugin lives entirely underexternal_plugins/preprocess-legal/and touches nothing else.Open question for maintainers
Carried over from the issue thread and still open: does a hook that intercepts document reads belong in a practice-area plugin, or closer to Cowork itself? It is built behind a clean interface so the trigger is swappable either way, and I am happy to move it if this is the wrong home.