[2810] Allow private config to override output directories and filename by evenmn · Pull Request #2811 · ecmwf/WeatherGenerator

evenmn · 2026-09-10T20:13:21Z

Description

Adds optional private_config overrides for model, log, and results directories, plus the output filename. Explicit directories are used directly, without appending the run ID. Existing defaults remain when overrides are omitted.

path_shared_working_dir: /work/weathergen

path_model: /models/deployed
path_logs: /var/log/weathergen
path_results: /forecasts
output_name: forecast.zip

When multiple samples are generated, rank can be used to separate the different samples:

output_name: "output_rank{rank:04d}.zip"

Other variables that can be used in the filename are epoch and step.

Checkpoint loading and logging use the selected private configuration.

Issue Number

Closes #2810

Is this PR a draft? Mark it as draft.

Checklist before asking for review

I have performed a self-review of my code
My changes comply with basic sanity checks:
- I have fixed formatting issues with ./scripts/actions.sh lint
- I have run unit tests with ./scripts/actions.sh unit-test
- I have documented my code and I have updated the docstrings.
- I have added unit tests, if relevant
I have tried my changes with data and code:
- I have run the integration tests with ./scripts/actions.sh integration-test
- (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
- (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
I have informed and aligned with people impacted by my change:
- for config changes: the MatterMost channels and/or a design doc
- for changes of dependencies: the MatterMost software development channel

FastEvaluation

I have updated the public documentation if necessary

clessig · 2026-09-10T20:27:42Z

@grassesi : could you also have a look?

clessig · 2026-09-10T20:27:58Z

@evenmn : could you lint please :)

clessig

Thanks, this is a useful feature. Some minor comments. We also should merge this into the preops branch.

Signed-off-by: evenmn <evenmn@fys.uio.no>

evenmn · 2026-09-11T08:37:19Z

@clessig Thank you for the comments. Please have a look at my answers and latest commits

clessig

Thanks for the changes! Some minor comments left. For the functions we anyway touch, we should be clear if config is "model config" or "private config". It makes the code much more readable.

clessig · 2026-09-12T11:01:45Z

+    run_id: str,
+    mini_epoch: int | None,
+    model_path: str | None,
+    *,


What is * for?

clessig · 2026-09-12T11:05:51Z

 def get_path_run(config: Config) -> Path:
    """Get the current runs results_path for storing run results and logs."""
-    return _get_shared_wg_path() / "results" / get_run_id_from_config(config)
+    return _get_path_output(config, "path_results", "results", get_run_id_from_config(config))


This function doesn't make sense--we don't the path for a run. Can you remove it. The function is used in trainer.py and in train_logger.py. It's calls should be replace by calls to get_path_logs()--but please check that you agree.

clessig · 2026-09-12T11:07:13Z


 def get_path_model(config: Config | None = None, run_id: str | None = None) -> Path:
-    """Get the current runs model_path for storing model checkpoints."""
+    """Get full_model_path if set, otherwise the shared per-run checkpoint directory."""


config is here private config, right? Can we change the variable name.

clessig · 2026-09-12T11:07:18Z

    ext = StoreType(config.zarr_store).value  # validate extension
    base_path = get_path_run(config)
    fname = f"validation_chkpt{mini_epoch:05d}_rank{config.rank:04d}.{ext}"
+    fname = config.get("output_name") or fname


config is here private config, right? Can we change the variable name.

Allow private config to override output directories and filename

81d6a21

github-project-automation Bot added this to WeatherGen-dev Sep 10, 2026

clessig requested a review from grassesi September 10, 2026 20:27

github-actions Bot added infra Issues related to infrastructure model Related to model training or definition (not generic infra) labels Sep 10, 2026

Avoid reassigning typed config parameter when loading a run

a42cf9b

clessig reviewed Sep 11, 2026

View reviewed changes

evenmn added 5 commits September 11, 2026 09:34

Rename exact checkpoint directory override to full_model_path

944b8e6

Clarify private configuration argument and variable names

5ebe17c

Renamed _get_output_directory to _get_path_output

1945007

Signed-off-by: evenmn <evenmn@fys.uio.no>

Resolve log directories in config before initializing loggers

27c6a08

Reverted 'loaded_config' to 'config' like it was before

07c1027

Signed-off-by: evenmn <evenmn@fys.uio.no>

clessig reviewed Sep 12, 2026

View reviewed changes

evenmn added 2 commits September 12, 2026 16:49

Add template formatting to output_name

2333e85

Use get_path_results as run results directory and simplify path helpers

42c4460

github-actions Bot added the eval anything related to the model evaluation pipeline label Sep 12, 2026

Uh oh!

Conversation

evenmn commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue Number

Checklist before asking for review

FastEvaluation

Uh oh!

clessig commented Sep 10, 2026

Uh oh!

clessig commented Sep 10, 2026

Uh oh!

clessig left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

evenmn commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clessig left a comment

Choose a reason for hiding this comment

Uh oh!

clessig Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

clessig Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

clessig Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

clessig Sep 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

evenmn commented Sep 10, 2026 •

edited

Loading

evenmn commented Sep 11, 2026 •

edited

Loading