Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Feat/read mitigations #426

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open

[WIP] Feat/read mitigations #426

wants to merge 33 commits into from

Conversation

rneher
Copy link
Member

@rneher rneher commented Apr 6, 2020

Related issues and PRs

preparatory PR for crowdsourced mitigation measures.

Description

Use data from countries and regions to prepopulate the scenarios with measures.

Impacted Areas in the application

only the data directory.

Testing

non yet.

ivan-aksamentov and others added 30 commits April 5, 2020 18:05
JSONSchemaStore is an abstract class, its fetch method needs to be
defined.

I've defined it in a Store class, so using new Store() is not
instanciating an abstract class anymore.

Also, with this change, we can now cross-reference things between
different schema files.
JSONSchemaStore is an abstract class, its fetch method needs to be
defined.

I've defined it in a Store class, so using new Store() is not
instanciating an abstract class anymore.

Also, with this change, we can now cross-reference things between
different schema files.
add quicktype and yaml-loader
@rneher rneher added the pr: wip label Apr 6, 2020
@vercel
Copy link

vercel bot commented Apr 6, 2020

This pull request is being automatically deployed with ZEIT Now (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

@rneher
Copy link
Member Author

rneher commented Apr 6, 2020

@alebrand just sketched this a bit. We'll have to see how we can align this with the JHU initiative.

@ivan-aksamentov ivan-aksamentov added pr: work in progress This is work in progress, do not merge until finished s:data Scope: related to data retrieval, parsing, transformation, storage, update t:feat Type: request of a new feature, functionality, enchancement and removed pr: wip labels Apr 6, 2020
@aitananeves
Copy link
Collaborator

@rneher I tried mapping the HIT categories to the covid19_scenario ones, could you have a look to the attached file? I made suggestions for extra fields (column C), otherwise many categories end up as "NA".
Note: I extracted the HIT categories from the csv download in the end, as from the codebook, I have the codes, but not the labels reported in the csv file.

We would end up with the following categories in covid19_scenarios:

  • Ban on mass gatherings
  • Nurseries/schools/universities closure
  • Restaurants/entertainment/shops closure
  • Household confinement
  • Contact Tracing
  • Borders closure/limitation of movements/symptoms screening
  • Isolation/quarantine of cases
  • Office closures
  • Universal mask
  • NA

HIT categories.xlsx

@deuteragenie
Copy link

@rneher I tried mapping the HIT categories to the covid19_scenario ones, could you have a look to the attached file? I made suggestions for extra fields (column C), otherwise many categories end up as "NA".
Note: I extracted the HIT categories from the csv download in the end, as from the codebook, I have the codes, but not the labels reported in the csv file.

We would end up with the following categories in covid19_scenarios:

  • Ban on mass gatherings
  • Nurseries/schools/universities closure
  • Restaurants/entertainment/shops closure
  • Household confinement
  • Contact Tracing
  • Borders closure/limitation of movements/symptoms screening
  • Isolation/quarantine of cases
  • Office closures
  • Universal mask
  • NA

[HIT categories.xlsx](https://github.com/neherlab/covid19_scenarios/files/4440775/HIT.categories.xls

There are many other types of measures or variations thereof. It would be good to collect an exhaustive list of those, together with a target reduction factor.

Other examples:

  • Outside activities allowed for max. 2 people, with distance > 2 meters
  • Nurseries open, but schools closed (so one need to distinguish between both)
  • Wearing of mask inside, outside, none, or both
    etc. etc. etc.

Ideally, people would package these individual measures into policy decision for each country / dates.

Issue is... some measures are interrelated.

Parser for JHU HIT mitigation measures. Still incomplete, I mention in mitigations_JHU.py some remaining Todos, to be discussed.
@ivan-aksamentov ivan-aksamentov added the IMPORTANT Take this immediately! label Apr 17, 2020
@ivan-aksamentov ivan-aksamentov added this to the 1.2 milestone Apr 17, 2020
@ivan-aksamentov
Copy link
Member

Hello!

After discussion with @rneher I put it to milestone 1.2:
https://github.com/neherlab/covid19_scenarios/milestone/3
and mark as important.

What help is needed on the engineering side to make this happen?

cc @nnoll

@ivan-aksamentov ivan-aksamentov mentioned this pull request Apr 17, 2020
36 tasks
@ivan-aksamentov ivan-aksamentov removed this from the 1.2 milestone May 1, 2020
@Vanuan
Copy link
Contributor

Vanuan commented May 13, 2020

Maybe for starters, measure values should be provided in the CSV rather than hardcoded from categories. It's highly unlikely that the same category of measures has the same effect in all countries. As there's a different level of consciousness/self-quarantining/implementation.

In some countries, the most accurate numbers is coming from death rates as there's 100% test coverage of deceased from respiratory illness. In others, the most accurate ones are serious illnesses as they test only hospitalized patients. Most have mixed strategy, so a manual fitting is always required.

@Vanuan
Copy link
Contributor

Vanuan commented May 13, 2020

For example, it's reasonable to assume that school closures would have a different effect based on demographics in each country.

@ivan-aksamentov
Copy link
Member

@Vanuan Thanks for your ideas. Yes, we planned to process JHU data for mitigation interventions, which has detailed information for different countries. After initial prototype with hardcoded data, we could then write a script for that and we would then automatically fetch and update the data every day, similarly to how we do with case counts.

But the entire adventure turned out to be quite tricky and in the end this thing stuck in an undetermined state...

@Vanuan
Copy link
Contributor

Vanuan commented May 14, 2020

Here's some content for inspiration: https://www.youtube.com/watch?v=BTLZu-1IMcE

I think for starters we could try gathering 3 sources of data:

  1. Manually crafted data of different mitigation measures for different territories (from wikipedia or elsewhere)
  2. Apple/Google mobility reports
  3. A set of values for manually fitting the prediction curve to actual data

Only after that we can start categorizing different mitigation measures and building a prediction model of their impact.

So, how about getting this file https://github.com/neherlab/covid19_scenarios/pull/426/files#diff-41177582a7c92a97242de0047786712d , removing all columns except the territory and the period, adding a column for manual mitigation value and using this piece while throwing out everything else:

https://github.com/neherlab/covid19_scenarios/pull/426/files#diff-9b3e50fca1abc2b9c6ba30b673fd737dR220-R232

@ivan-aksamentov ivan-aksamentov removed IMPORTANT Take this immediately! pr: work in progress This is work in progress, do not merge until finished s:data Scope: related to data retrieval, parsing, transformation, storage, update t:feat Type: request of a new feature, functionality, enchancement labels Jul 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants