Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event set flood damages #177

Merged
merged 24 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8bdfe19
First stab at event-set flood damages
thomas-fred Feb 7, 2024
4b079da
Update logging string
thomas-fred Feb 7, 2024
897790e
Update test rule name
thomas-fred Feb 7, 2024
d11d848
Remove unnecessary constant from config
thomas-fred Feb 7, 2024
7f9e503
Fetch rehabilitation costs during damage calculation
thomas-fred Feb 8, 2024
7f697e5
Label edges, not entire network
thomas-fred Feb 8, 2024
cae8bea
Add additional damage curves
thomas-fred Feb 8, 2024
479c832
Ensure cost is a number not an array
thomas-fred Feb 9, 2024
9aace8a
Merge branch 'feature/event_set_flood_damages' of github.com:nismod/o…
thomas-fred Feb 9, 2024
8409f4b
Update rehabilitation costs
thomas-fred Feb 9, 2024
1022538
Use motorway specific damage function
thomas-fred Feb 12, 2024
bb27beb
Add concatenation rule
thomas-fred Feb 12, 2024
4801815
Set rail bridge asset_type correctly
thomas-fred Feb 12, 2024
47fa4d3
Merge branch 'feature/event_set_flood_damages' of github.com:nismod/o…
thomas-fred Feb 12, 2024
a2d23ab
Use a smaller extent raster for testing trimming
thomas-fred Feb 16, 2024
1af4d6a
Consolidate damage curves and rehab costs
thomas-fred Feb 16, 2024
6a22ec1
Employ new `asset_type` categories
thomas-fred Feb 16, 2024
98f425e
Update test input and output
thomas-fred Feb 16, 2024
49c6144
Update network creation tests input and output
thomas-fred Feb 16, 2024
f5180d3
Concat for damage fractions, too
thomas-fred Feb 16, 2024
9e198de
Remove comment
thomas-fred Feb 16, 2024
94e9fe9
Add event set flooding test
thomas-fred Feb 16, 2024
047be28
Factor out damages code common to events & RPs
thomas-fred Feb 16, 2024
679a5e8
Empty commit, rerun CI
thomas-fred Feb 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions bundled_data/README.md

This file was deleted.

1 change: 0 additions & 1 deletion bundled_data/damage_curves/flood/rail_railway.csv

This file was deleted.

1 change: 0 additions & 1 deletion bundled_data/damage_curves/flood/road_paved.csv

This file was deleted.

277 changes: 0 additions & 277 deletions bundled_data/storm_basins.geojson

This file was deleted.

Binary file removed bundled_data/transport/rehabilitation.xlsx
Binary file not shown.
Binary file removed bundled_data/transport/speeds.xlsx
Binary file not shown.
Binary file removed bundled_data/transport/tariffs.xlsx
Binary file not shown.
84 changes: 41 additions & 43 deletions config/README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,41 @@
# Configuration

The pipeline is configured from `config/config.yaml` file.
We provide a basic config file `config/config.yml` with explanatory comments that you can modify.

The configuration file is meant to specify the location of input data
and outputs, as well as some runtime settings.

- `hazard_datasets`: Named list of file paths to `.txt` files containing a list of hazard files.
Files can be specified (both the `.txt` files and the `.tif` files they point to) either as filenames
relative to the project root, as absolute file paths, or as remote resources (`http://` or `https://`).
The names in the list should not include `_` or `/` characters.
Remote resources will be fetched with the `wget` utility.
- `infrastructure_datasets`: Named list of file paths to `.osm.pbf` files to use as datasets.
These can be local files, specified with absolute file paths or file paths relative to the project root,
or they can be remote files to fetch with `wget`.

- `slice_count`: Number of slices to take for each infrastructure dataset.
More slices allows for greater parallelization, but will also duplicate sections of roads that cross
slice boundaries, so too high a number can lead to redundancy.
- `keep_tags`: Osmium tags to preserve in .geoparquet file.
- `osmium_tags_filters_file`: File containing the OSM attributes to filter the input data

Modifying the configuration file on the whole will *not* trigger a re-run of
the pipeline by snakemake. If you wish to rerun the whole pipeline after
altering the configuration, use:

```
snakemake --cores all --forceall
```

Re-running the whole pipeline from the start might not be necessary. For
instance if you modify `<output_dir>`, only the last few pipeline steps will be
altered. In this case, you can ask snakemake to (re)start from the first
affected rule (see `Snakefile`), and it will figure out what must be done to
complete the pipeline. In this case:

```
snakemake --cores all -R intersection
```

which will re-run `intersection` and `join_data` rules,
in this order.
# open-gira configuration directory

## README.md
This file.

## config.yaml
YAML configuration file. This file contains parameters that are frequently
modified prior to an analysis. Refer to it for more information.

## land_use_to_surface_roughness.csv
A lookup table from a land use classification scheme to a typical surface
roughness length. This is used in downscaling wind speeds in tropical-cyclone
analyses.

## arc_cluster
`snakemake` profile for running open-gira on the University of Oxford's ARC
cluster.

## damage_curves
CSV files, one per `asset_type`, describing the relationship between hazard
intensity and damage fraction. These are used for direct damage estimation.

## hazard_resource_locations
Fairly self-explanatory, contains files containing lists of hazard file URLs to
acquire for analyses.

## osm_filters
Text files for use by osmium to [filter features based on
tag](https://docs.osmcode.org/osmium/latest/osmium-tags-filter.html). We used
these to construct infrastructure networks from OpenStreetMap data.

## rehab_costs
CSV files containing rehabilitation cost estimates in USD. One file per sector,
each row a unique `asset_type`.

## storm_sets
JSON files, each containing a single list. The list may contain zero or many
unique string storm IDs. These can be used to subset a parent `STORM_SET`, e.g.
IBTrACS may be subset to contain just storms that hit Mexico since 2000, if the
relevant storm IDs are provided. Where the list provided is empty, this is
interpreted as "process every storm in the parent set", i.e. all of IBTrACS.
Loading
Loading