Skip to content

Forecast Checks

Jarad Niemi edited this page Jun 4, 2020 · 18 revisions
- header must include `location`, `target`, `type`, `quantile`, `value` (required for zoltpy) and `forecast_date`, `target_end_date`
- each row must have the same number of columns as header
- `location` must be in [locations.csv](https://github.com/reichlab/covid19-forecast-hub/blob/master/data-locations/locations.csv)
- `target` must be in 
    ```r
    paste(0:130, "day ahead inc death")
    paste(0:130, "day ahead cum death")
    paste(1:20,  "wk ahead inc death")
    paste(1:20,  "wk ahead cum death")
    paste(0:130, "day ahead inc hosp")
    ```
- the # in "# day ahead" or "# week ahead" must be integers (redundant)
- `forecast_date` and `target_end_date` must be in YYYY-MM-DD format
- `quantile` must be in 
    ```r
    c(0.01, 0.025, seq(0.05, 0.95, by = 0.05), 0.975, 0.99)
    ```
- checks `quantile` must be an int or float in [0, 1]
- checks `value` must be an int or float
  1. validates date alignment as documented in the issue add additional validations

  2. validates quantiles and values (i.e,. at the prediction level):

    • checks that entries in value must be non-decreasing as quantiles increase
    • checks that elements in the quantile are unique
  3. validates quantiles as a group:

    • there must be exactly one point prediction for each location/target pair