Describe the task. Describe the task. It can be a feature, a set of experiments, documentation, etc.
Problem
For hydrological applications, long-term memory is critical to track state variables such as snowpack and snow water equivalent (SWE), which are poorly represented in existing datasets and must be estimated via accumulation from the start of the hydrological year.
The current WeatherGenerator operates at a 6-hour temporal resolution. Even with rollout training, the model can only retain information over relatively short horizons (days), making it unsuitable for long-term state accumulation.
Proposed approach
Inspired by long-term memory mechanisms used in hydrology (e.g., LSTMs) and recent work such as Griffin (https://arxiv.org/abs/2402.19427), we propose adding an explicit long-term memory component to the forecast engine that complements the model’s existing short-term capabilities.
Implementation outline
- Forecast engine with latent accumulation: Maintain and update a persistent latent state across forecast steps to enable long-term memory.
- Latent spin-up (no prediction): Initialize the latent state via a spin-up phase (e.g., from the start of the hydrological year) without generating forecasts.
- Hydrology data reader: Data reader that supports the de facto standard hydrology data format (e.g. Caravan)
Data sources
The hydrology dataset that we use here will follow the standard introduced by Kratzert et al., 2019 (https://www.nature.com/articles/s41597-023-01975-w), which consist of the following components:
- Time series data (ERA5 forcings and streamflow)
- Static basin attributes (catchment area, location, shape factor, main channel length, ...)
- Geospatial metadata (climate, topographic, surface, soil, geology, ...)
- (consistent identifiers)
We will use the global Caravan dataset (16,000 catchments) for pretraining and the NVE dataset (hundreds of catchments) for finetuning. The NVE dataset is found to have a higher overall quality than the Caravan dataset. References:
Status
Proposal / design discussion, feedback on feasibility and design choices welcome.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
URL to the design document
No response
Area
Describe the task. Describe the task. It can be a feature, a set of experiments, documentation, etc.
Problem
For hydrological applications, long-term memory is critical to track state variables such as snowpack and snow water equivalent (SWE), which are poorly represented in existing datasets and must be estimated via accumulation from the start of the hydrological year.
The current WeatherGenerator operates at a 6-hour temporal resolution. Even with rollout training, the model can only retain information over relatively short horizons (days), making it unsuitable for long-term state accumulation.
Proposed approach
Inspired by long-term memory mechanisms used in hydrology (e.g., LSTMs) and recent work such as Griffin (https://arxiv.org/abs/2402.19427), we propose adding an explicit long-term memory component to the forecast engine that complements the model’s existing short-term capabilities.
Implementation outline
Data sources
The hydrology dataset that we use here will follow the standard introduced by Kratzert et al., 2019 (https://www.nature.com/articles/s41597-023-01975-w), which consist of the following components:
We will use the global Caravan dataset (16,000 catchments) for pretraining and the NVE dataset (hundreds of catchments) for finetuning. The NVE dataset is found to have a higher overall quality than the Caravan dataset. References:
Status
Proposal / design discussion, feedback on feasibility and design choices welcome.
Hedgedoc URL, if you are keeping notes, plots, logs in hedgedoc.
No response
URL to the design document
No response
Area