feat: add SMS short-text de-identification#1841
Conversation
9484669 to
6a3317e
Compare
|
Ready to merge. I rebased both commits onto current Validation is complete: the focused SMS suite passes, the full local suite passes (6,045 passed, 48 skipped), formatting/lint/type checks, all changed-file pre-commit hooks, strict docs, and package build pass. Hosted CI is fully green across Linux, macOS, Windows, security/SBOM/policy checks, and amd64/arm64 container smoke tests; the one macOS timing flake cleared on rerun without code changes. The diff and PR metadata are clean of prohibited vendor/assistant names and co-author trailers. This closes #1429. |
Description
Adds an SMS-scale
short_textde-identification preset and export adapters for RapidPro-shaped JSON and generic CSV helpdesk logs. The path preserves message offsets and source schemas, pseudonymizes contact values, coarsens timestamps, and processes large record streams in bounded batches.Type of Change
Changes Made
Testing
master: 6,045 passed, 48 skipped.Additional validation:
make formatmake lintmake format-checkmake type-checkpre-commit run --files <changed files>make docs-buildpython3 -m buildDocumentation
CHANGELOG.mdis unchanged; release notes can include this feature when it ships.Code Quality
Dependencies
Checklist
Related Issues
Closes #1429.
Screenshots/Examples
See
examples/sms_deid_helpdesk_logs.pyanddocs/sms-short-text-deid.mdfor runnable JSON and CSV usage.