Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider automating future schema updates for covid_hosp #747

Open
krivard opened this issue Oct 21, 2021 · 2 comments
Open

Consider automating future schema updates for covid_hosp #747

krivard opened this issue Oct 21, 2021 · 2 comments

Comments

@krivard
Copy link
Contributor

krivard commented Oct 21, 2021

Schema updates for covid_hosp are a huge pain, since there are three sets of 100+ column names to diff, many of which need to have separate spellings in SQL to comply with stricter naming requirements, plus different spellings of data types in SQL vs Python. Consider automating this procedure using a similar setup to what we have for advanced meta.

ah! I mean this: https://github.com/cmu-delphi/delphi-epidata/blob/dev/.github/workflows/update_gdocs_data.yml

ie

  1. pull down a copy of the csv from the spreadsheet
  2. sanitize the bejeezus out of it to make sure we don't end up in a Bobby Tables situation
  3. parse it into covid_hosp.sql and database.py x3 automatically

Originally posted by @krivard in #720 (comment)

@krivard
Copy link
Contributor Author

krivard commented Sep 8, 2022

...I found my old half-finished code. It is not very good, but maybe it will help you get started:

covid_hosp_auto.zip

@krivard
Copy link
Contributor Author

krivard commented Jan 18, 2023

Cleaner but still incomplete version in progress at krivard/covid_hosp_auto

Probably best to start with the auto-updater section in the README.

Unit tests for this work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants