Add configurable overlap to the document splitter

[jzonthemtn]

Background

Phileas already splits large documents before filtering: when config.splitting is enabled and the input length crosses the configured threshold, PlainTextFilterService uses a SplitService (CharacterCountSplitService, LineWidthSplitService, or NewLineSplitService) to break the document into chunks and filters each chunk independently.

The CharacterCountSplitService is sentence-aware (it splits on sentence-ending punctuation), so most entities are not cut. But there is no overlap between adjacent chunks, so an entity that does straddle a split boundary can be partially seen in each chunk and missed by both. In redaction terms, a missed entity is a leak.

Proposal

Add a configurable overlap to the splitter so adjacent chunks share a small window of text, and reconcile spans detected in the overlap so they are not duplicated or double-counted.

• A config.splitting option for overlap size (characters), default 0 to preserve current behavior.
• When overlap is set, each chunk includes the trailing N characters of the previous chunk.
• Spans detected within the overlap region are de-duplicated/merged so a single entity at a seam is reported once, with correct document offsets.

Acceptance Criteria

• config.splitting accepts an overlap setting (characters), defaulting to 0 (no behavior change when unset).
• The splitter produces overlapping chunks when overlap > 0.
• Spans found in an overlap region are de-duplicated and mapped back to correct document offsets (no double redaction, no shifted offsets).
• Tests cover an entity that spans a chunk boundary, confirming it is detected once with the overlap enabled and (documented) missed without it.
• Phileas docs/docs/ updated to document the new splitting option.
• No em-dashes in the content (org convention).

Related

• Split out of a broader streaming/chunked-processing effort that was closed as over-scoped. This is the one focused, well-scoped correctness refinement from that issue: chunk overlap so entities spanning a split boundary are not missed. The broader streaming/backpressure asks are handled at the connector layer and are not pursued here.