Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi level hydrofabric #497

Closed
wants to merge 44 commits into from
Closed

Multi level hydrofabric #497

Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
bc9884d
Add trival multi layer hydrofabric sample.
donaldwj May 11, 2022
0b5acf6
Added Multlevel hydrofabric sample file.
donaldwj Jun 6, 2022
ae1f8a9
Added level variable to HY_Catchment. Added function to get and set c…
donaldwj Sep 28, 2022
32bb806
Add missing config file for a multilevel configuration testing.
donaldwj Sep 28, 2022
88434af
Updated level variables in multilevel hydrofabric test to match docum…
donaldwj Sep 28, 2022
758898f
Update non mpi code to handle hydro fabric multiple levels. Fix bug w…
donaldwj Oct 5, 2022
6f6413e
Add support for hydrofabric levels.
donaldwj Nov 2, 2022
6eb6398
Added runtime loop based on levels.
donaldwj Nov 2, 2022
f212291
saving for merge
donaldwj Feb 15, 2023
c813e1e
Fix for error in rebase.
donaldwj Feb 15, 2023
79665b9
Fix missing variable in HY_Features.hpp
donaldwj Feb 15, 2023
b82da39
added test file using google test replacing stand alone test.
donaldwj Mar 1, 2023
29216b7
fix parsing of layer data from the formulation config file.
donaldwj Mar 1, 2023
dbf2b27
Update test code for multilayer.
donaldwj Mar 1, 2023
652e5ec
Multilayer test added to CMake.
donaldwj Mar 1, 2023
922e3c5
Add datastructures for storing layer meta data and add those structur…
donaldwj Mar 3, 2023
42e0a5e
Inital multi time step muti level time loop
donaldwj Mar 7, 2023
bb95a5a
Simulation_Time methods useful for multilayer
Mar 9, 2023
f1d0628
Change get_properties() of the FeatureBase class to return const or n…
donaldwj Mar 15, 2023
605810a
add an internal map to the network class to track layer association.
donaldwj Mar 15, 2023
e1881ef
Add support for filtering a network based on level.
donaldwj Mar 17, 2023
6c623cb
Create inital skelton of layer class.
donaldwj Mar 22, 2023
01539c5
add <string> to simulation_time.hpp to fix VSCode issue
donaldwj Mar 22, 2023
563fa87
Add missing changes to Layer.hpp
donaldwj Mar 29, 2023
726fc4e
moved logic to run catchments into layer class
donaldwj Apr 5, 2023
7b1ed93
remove code that is no longer needed
donaldwj Apr 5, 2023
2d067fa
Fix typos
donaldwj Apr 5, 2023
062586d
update layer control for better depnd btwn layers
donaldwj Apr 5, 2023
2867f03
Add command to advance the current time stored in a layer.
donaldwj Apr 5, 2023
1c892bf
add current_timestep_epoch_time to layers
donaldwj Apr 5, 2023
e119816
commit changes to simulation time object to allow current time update.
donaldwj Apr 11, 2023
3f0299e
Fix to missing changes on pervious commit.
donaldwj Apr 11, 2023
b16bb3b
Updates to fix testing on Multilayer
donaldwj Apr 20, 2023
dc7b05f
Add current_time_index to SurfaceLayer to
donaldwj Apr 20, 2023
e7bcd87
Added more documentation to main time loop.
donaldwj Apr 27, 2023
9c00898
Fix to HY_Features constructor
donaldwj Apr 27, 2023
96fdee2
Add defualt layer creation to formulation manager
donaldwj Apr 28, 2023
c9fe2d6
fix bad units on defualt layer
donaldwj Apr 28, 2023
1cb583e
Correct units string for default layer.
donaldwj May 5, 2023
52d02b8
Fix merge error in HH_Features_MPI.hpp
donaldwj May 5, 2023
acdd022
Required fixes for serial build
mattw-nws Jul 6, 2023
b64af75
Create HY_Catchments with level metadata in single processor mode.
donaldwj Jul 14, 2023
22a7c93
Change HY_Features constructor to use the link hydrofabric version th…
donaldwj Jul 14, 2023
4b65acc
Apply spelling/grammar/clarity suggestions from code review
mattw-nws Jul 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@ CMakeCache.txt
/cmake-build-debug
/cmake_build
/build
/cmake_build*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this PR messing with .gitignore?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Common problem I find myself caught in. Pro tip: https://stackoverflow.com/a/1753078 ... I've started adding my "personal" Git ignore patterns to .git/info/exclude instead.


# Static Headers
include/bmi.hpp
Expand Down
11 changes: 11 additions & 0 deletions data/catchment_data_multilayer.geojson
View file
Open in desktop

Large diffs are not rendered by default.

196 changes: 196 additions & 0 deletions data/example_multilevel_realization_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
{
"global": {
"formulations": [
{
"name": "tshirt_c",
"params": {
"maxsmc": 0.439,
"wltsmc": 0.066,
"satdk": 0.00000338,
"satpsi": 0.355,
"slope": 1.0,
"scaled_distribution_fn_shape_parameter": 4.05,
"multiplier": 0.0,
"alpha_fc": 0.33,
"Klf": 0.01,
"Kn": 0.03,
"nash_n": 2,
"Cgw": 0.01,
"expon": 6.0,
"max_groundwater_storage_meters": 1.0,
"nash_storage": [
0.0,
0.0
],
"soil_storage_percentage": 0.667,
"groundwater_storage_percentage": 0.5,
"timestep": 3600,
"giuh": {
"giuh_path": "./test/data/giuh/GIUH.json",
"crosswalk_path": "./data/crosswalk.json"
}
}
}
],
"forcing": {
"file_pattern": ".*{{id}}.*.csv",
"path": "./data/forcing/"
}
},
"time": {
"start_time": "2015-12-01 00:00:00",
"end_time": "2015-12-30 23:00:00",
"output_interval": 3600
},
"catchments": {
"cat-27": {
"formulations": [
{
"name":"tshirt_c",
"params": {
"maxsmc": 0.439,
"wltsmc": 0.066,
"satdk": 0.00000338,
"satpsi": 0.355,
"slope": 1.0,
"scaled_distribution_fn_shape_parameter": 4.05,
"multiplier": 0.0,
"alpha_fc": 0.33,
"Klf": 0.01,
"Kn": 0.03,
"nash_n": 2,
"Cgw": 0.01,
"expon": 6.0,
"max_groundwater_storage_meters": 1.0,
"nash_storage": [
0.0,
0.0
],
"soil_storage_percentage": 0.667,
"groundwater_storage_percentage": 0.5,
"timestep": 3600,
"giuh": {
"giuh_path": "./test/data/giuh/GIUH.json",
"crosswalk_path": "./data/crosswalk.json",
"cdf_ordinates": [
0.1,
0.35,
0.2,
0.14,
0.1,
0.06,
0.05
]
}
}
}
],
"forcing": {
"path": "./data/forcing/cat-27_2015-12-01 00_00_00_2015-12-30 23_00_00.csv"
}
},
"cat-52": {
"formulations": [
{
"name": "simple_lumped",
"params": {
"sr": [
1.0,
1.0,
1.0
],
"storage": 1.0,
"gw_storage": 1.0,
"gw_max_storage": 10.0,
"nash_max_storage": 2.0,
"smax": 5,
"a": 1.0,
"b": 10.0,
"Ks": 0.1,
"Kq": 0.01,
"n": 3,
"t": 0
}
}
],
"forcing": {
"path": "./data/forcing/cat-52_2015-12-01 00_00_00_2015-12-30 23_00_00.csv"
}
},
"cat-67": {
"formulations": [
{
"name":"tshirt_c",
"params": {
"maxsmc": 0.439,
"wltsmc": 0.066,
"satdk": 0.00000338,
"satpsi": 0.355,
"slope": 1.0,
"scaled_distribution_fn_shape_parameter": 4.05,
"multiplier": 0.0,
"alpha_fc": 0.33,
"Klf": 0.01,
"Kn": 0.03,
"nash_n": 2,
"Cgw": 0.01,
"expon": 6.0,
"max_groundwater_storage_meters": 1.0,
"nash_storage": [
0.0,
0.0
],
"soil_storage_percentage": 0.667,
"groundwater_storage_percentage": 0.5,
"timestep": 3600,
"giuh": {
"giuh_path": "./test/data/giuh/GIUH.json",
"crosswalk_path": "./data/crosswalk.json",
"cdf_ordinates": [
0.1,
0.35,
0.2,
0.14,
0.1,
0.06,
0.05
]
}
}
}
],
"forcing": {
"path": "./data/forcing/cat-67_2015-12-01 00_00_00_2015-12-30 23_00_00.csv"
}
},
"agg-1": {
"formulations": [
{
"name": "bmi_fortran",
"params": {
"model_type_name": "bmi_fortran_noahowp",
"library_file": "./extern/noah-owp-modular/cmake_build/libsurfacebmi.so",
"forcing_file": "",
"init_config": "./data/bmi/fortran/noah-owp-modular-init-agg-1.namelist.input",
"allow_exceed_end_time": true,
"main_output_variable": "QINSUR",
"variables_names_map": {
"PRCPNONC": "atmosphere_water__liquid_equivalent_precipitation_rate",
"Q2": "atmosphere_air_water~vapor__relative_saturation",
"SFCTMP": "land_surface_air__temperature",
"UU": "land_surface_wind__x_component_of_velocity",
"VV": "land_surface_wind__y_component_of_velocity",
"LWDN": "land_surface_radiation~incoming~longwave__energy_flux",
"SOLDN": "land_surface_radiation~incoming~shortwave__energy_flux",
"SFCPRS": "land_surface_air__pressure"
},
"uses_forcing_file": false
}
}
],
"forcing": {
"path": "./data/forcing/agg-1_2015-12-01 00_00_00_2015-12-30 23_00_00.csv"
}
}
}
}
36 changes: 25 additions & 11 deletions include/core/HY_Features.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#define HY_FEATURES_H

#include <unordered_map>
#include <set>

#include <HY_Catchment.hpp>
#include <HY_HydroNexus.hpp>
Expand Down Expand Up @@ -65,18 +66,7 @@ namespace hy_features {
* @param network
* @param formulations
*/
HY_Features( network::Network network, std::shared_ptr<Formulation_Manager> formulations);
donaldwj marked this conversation as resolved.
Show resolved Hide resolved

/**
* @brief Construct a new HY_Features object From a GeoJSON feature collection and a set of formulations.
donaldwj marked this conversation as resolved.
Show resolved Hide resolved
*
* Constructs the network::Network index from the GeoJSON feature collection and link_key and then
* \copydetails HY_Features::HY_Features(network::Network,std::shared_ptr<Formulation_Manager>)
*
* @param catchments
* @param link_key
* @param formulations
*/
HY_Features( geojson::GeoJSON catchments, std::string* link_key, std::shared_ptr<Formulation_Manager> formulations);

/**
Expand Down Expand Up @@ -116,6 +106,21 @@ namespace hy_features {
*/
inline auto catchments(){return network.filter("cat");}

/**
* @brief An iterator of only the catchment feature ids from only the specified layer
*
* @return auto
*/
inline auto catchments(long lv) {
return network.filter("cat",lv);
}

/**
* @brief Return a set of levels that contain a catchment
*/

inline const auto& levels() { return hf_levels; }

/**
* @brief An iterator of only the nexus feature ids
*
Expand Down Expand Up @@ -180,6 +185,8 @@ namespace hy_features {

private:

void init();

/**
* @brief Internal mapping of catchment id -> HY_Catchment pointer.
*
Expand All @@ -204,6 +211,13 @@ namespace hy_features {
*/
std::shared_ptr<Formulation_Manager> formulations;

/**
* @brief The set of levels that contain at least one catchment
*/
std::set<long> hf_levels;

geojson::GeoJSON fabric;

};
}

Expand Down
12 changes: 12 additions & 0 deletions include/core/HY_Features_MPI.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#ifdef NGEN_MPI_ACTIVE

#include <unordered_map>
#include <set>

#include <HY_Catchment.hpp>
#include <HY_PointHydroNexusRemote.hpp>
Expand Down Expand Up @@ -32,6 +33,16 @@ namespace hy_features {
inline bool is_remote_sender_nexus(const std::string& id) {
return _nexuses.find(id) != _nexuses.end() && _nexuses[id]->is_remote_sender();
}

inline auto catchments(long lv) {
mattw-nws marked this conversation as resolved.
Show resolved Hide resolved
return network.filter("cat",lv);
}

/**
* @brief Return a set of levels that contain a catchment
*/

inline const auto& levels() { return hf_levels; }

inline std::vector<std::shared_ptr<HY_HydroNexus>> destination_nexuses(const std::string& id) {
std::vector<std::shared_ptr<HY_HydroNexus>> downstream;
Expand Down Expand Up @@ -77,6 +88,7 @@ namespace hy_features {
std::unordered_map<std::string, std::shared_ptr<HY_PointHydroNexusRemote>> _nexuses;
network::Network network;
std::shared_ptr<Formulation_Manager> formulations;
std::set<long> hf_levels;
int mpi_rank;
int mpi_num_procs;

Expand Down
Loading