Rr 26 add schemas#29
Conversation
There was a problem hiding this comment.
Pull request overview
Adds Pandera dataframe schema modules (with mock data generators) for STF datasets and wires the param_estimates dataset config to a schema file, along with a dependency update.
Changes:
- Add new schema modules under
cfa/catalog/public/datasets/stf/schemas/(e.g.,nwss,nssp_gold,nhsn_hrd,param_estimates) - Reference the new
param_estimatesschema fromcfa/catalog/public/datasets/stf/param_estimates.toml - Update
pyproject.tomldependencies (notablyfaker)
Reviewed changes
Copilot reviewed 5 out of 7 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Updates project dependencies (adds an extra faker entry). |
| cfa/catalog/public/datasets/stf/schemas/param_estimates.py | Introduces a Pandera schema + mock data generator for param_estimates. |
| cfa/catalog/public/datasets/stf/schemas/nwss.py | Introduces a Pandera schema + mock data generator for nwss. |
| cfa/catalog/public/datasets/stf/schemas/nssp_gold.py | Introduces a Pandera schema + mock data generator for nssp_gold. |
| cfa/catalog/public/datasets/stf/schemas/nhsn_hrd.py | Introduces Pandera schemas + mock data generators for nhsn_hrd extract/load. |
| cfa/catalog/public/datasets/stf/param_estimates.toml | Adds a schemas pointer to the new param_estimates schema module. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…alog-pub into rr-26-add-schemas
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 7 changed files in this pull request and generated 11 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 7 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@ryanraaschCDC Is there some documentation about how I can use this? |
|
there is documentation captured in cfa-dataops. I can add documentation here if you think that is helpful. |
|
I added some documentation for using schemas and mock data |
No description provided.