Skip to content

Add structured and tabular data redaction #319

Description

@jzonthemtn

Summary

Phileas is built around free text. It does not have a first-class path for structured and semi-structured data (CSV, database tables, JSON) where redaction should be column- or field-aware. Microsoft Presidio offers this via presidio-structured. This issue tracks adding structured/tabular support to Phileas.

Surfaced while writing the Phileas vs Presidio comparison, where we concede structured/tabular data to Presidio.

Why it matters

A large share of PII lives in tables and JSON, not prose: exports, warehouse columns, event payloads, API logs. Field-aware redaction (detect that a column is SSN, apply a strategy to the whole column consistently) is materially different from running free-text detection over a serialized blob, and it pairs naturally with the AI training-data and analytics use cases where consistent pseudonymization matters. Without it, structured-data teams have a reason to pick Presidio.

Acceptance Criteria

  • Phileas can accept structured input (at minimum tabular/CSV and JSON) and apply redaction per field or column rather than only as free text
  • Detection can be column-aware: infer the PII type of a column from a sample and/or accept an explicit field-to-entity mapping
  • Field-level strategies reuse the existing policy model and strategies (redact, mask, FPE, consistent pseudonymization), so a value pseudonymized in one column maps consistently where intended
  • Consistency across rows is supported (the same input value maps to the same token within a configured scope), which is the key requirement for analytics and training data
  • The capability is exposed in at least the primary (Java) runtime with a documented path for Python
  • User documentation under docs/docs/ covers structured/tabular redaction with a runnable example
  • The Phileas vs Presidio comparison and product page are updated once shipped

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions