Skip to content

Add a synthetic tabular-data generator preserving column distributions#1760

Open
maziyarpanahi wants to merge 1 commit into
masterfrom
feature/om-587-synthetic-tabular-data-generator
Open

Add a synthetic tabular-data generator preserving column distributions#1760
maziyarpanahi wants to merge 1 commit into
masterfrom
feature/om-587-synthetic-tabular-data-generator

Conversation

@maziyarpanahi

Copy link
Copy Markdown
Owner

Description

Adds a local, dependency-free synthetic tabular-data generator that preserves empirical column distributions and numeric rank correlations while preventing exact source-row copies.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Test addition/improvement

Changes Made

  • Fit reusable numeric and categorical marginal profiles with pairwise numeric rank correlations.
  • Generate deterministic seeded samples using stratified empirical quantiles and a Gaussian rank-correlation model.
  • Reject exact source-row fingerprints and fail safely when the available support cannot produce unseen rows.
  • Report KS/total-variation marginal fidelity, correlation deltas, an overall score, and copied-row count.

Testing

  • Added unit tests covering marginals, correlation fidelity, reproducibility, source-row exclusion, missing values, categorical edge cases, and invalid inputs.
  • .venv/bin/python -m pytest tests/ -q — 5,211 passed, 46 skipped.
  • make format, make lint, and make format-check.
  • Scoped pre-commit hooks.

Documentation

  • Added public API docstrings and documented the default fidelity tolerances.
  • Updated CHANGELOG.md (not required for this scoped feature PR).

Dependencies

  • No new dependencies.

Related Issues

Closes #945

@maziyarpanahi maziyarpanahi added roadmap-v2 OpenMed V2 roadmap backlog feature New capability P2 Medium labels Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New capability P2 Medium roadmap-v2 OpenMed V2 roadmap backlog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a synthetic tabular-data generator preserving column distributions

1 participant