Skip to content

Conversation

tlyon3
Copy link
Collaborator

@tlyon3 tlyon3 commented Jul 30, 2020

Most of the data is at the state level. The more granular data is separated into different districts, where a district could have 1 or more counties grouped. I'm not sure how to store this data

@tlyon3 tlyon3 requested review from cc7768 and sglyon July 30, 2020 22:09
@tlyon3
Copy link
Collaborator Author

tlyon3 commented Aug 5, 2020

@cc7768 @sglyon , this is ready. The failure is coming from the FloridaHospitals scraper

Copy link
Contributor

@cc7768 cc7768 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one looked like it was a bit tricky -- Nice work getting it all.

One unfortunate thing is that it doesn't seem like Utah is choosing to report much of the information that we're prioritizing now.

I'd vote to leave this on the back burner for now (and cross our fingers that they choose to make a more useful dashboard) and focus on on our short-term test coverage goals. Thoughts @sglyon @tlyon3 ?

return (
sorts[["dt", "negative_tests_total", "positive_tests_total"]]
.melt(id_vars=["dt"], var_name="variable_name")
.assign(vintage=pd.Timestamp.utcnow(), fips=self.state_fips)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a new method for fetching the vintage so that we are consistent in how we set the vintage. See DatasetBaseNoDate._retrieve_vintage

.agg("sum")
.reset_index()[["dt", "cases_total", "cumulative_hospitalized"]]
.melt(id_vars=["dt"], var_name="variable_name")
.assign(vintage=pd.Timestamp.utcnow(), fips=self.state_fips)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other vintage comment

}
)
renamed["dt"] = (
renamed["dt"].map(lambda x: pd.datetime.fromtimestamp(x / 1000)).dt.date
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the ArcGIS._esri_ts_to_dt method rather than do this by hand.

"DISTNAME": "district",
"COVID_Cases_Total": "cases_total",
"Day": "dt",
"Hospitalizations": "cumulative_hospitalized",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't currently collect cumulative hospitalization data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants