Skip to content

Iss615/read ancils in time windows - #735

Draft
Ricky Olivier (ricky-lv426) wants to merge 19 commits into
MetOffice:mainfrom
ricky-lv426:iss615/read_ancils_in_time_windows
Draft

Ricky Olivier (ricky-lv426) wants to merge 19 commits into
MetOffice:mainfrom
ricky-lv426:iss615/read_ancils_in_time_windows

Conversation

@ricky-lv426

Copy link
Copy Markdown
Contributor

PR Summary

Sci/Tech Reviewer:
Code Reviewer:

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

@ricky-lv426

Copy link
Copy Markdown
Contributor Author

Summary of changes done so far:

  • Added initial ancillary-collection wiring for migration groundwork.
  • Switched SST and aerosol ancils to use dedicated field collections.
  • Completed follow-up fixes in SST/aerosol file registration.
  • Added guard logic to avoid hard abort when collections were unavailable during early init_io.
  • Ensured ancillary collections are available in initialization flow.
  • Refactored SST/aerosol migration path for clearer integration and robustness.

How the changes try to be functionally equivalent to old code:

  • For SST and aerosol, migration preserves functional behavior by keeping the same model fields and update intent, while moving temporal ownership from legacy time_axis-driven updates to file-owned timeseries handling.
  • SST post-processing into tile_temperature is explicitly preserved via tstar_sea availability checks and update call placement independent of legacy sst_time axis.
  • Non-migrated ancils continue to run through the existing legacy ancil_times_list driver path, so branch behavior remains mixed-mode rather than all-or-nothing.

@ricky-lv426
Ricky Olivier (ricky-lv426) force-pushed the iss615/read_ancils_in_time_windows branch from 189635f to d5a1575 Compare October 8, 2026 09:09
@ricky-lv426

Ricky Olivier (ricky-lv426) commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Update of changes done so far:

Summary

The aerosol climatology ancillaries (glomap_mode = climatology or
dust_and_clim) are read through the new lfric_xios_file_type /
temporal_type API in lfric_core when ancil_option = 'updating'.
In fixed mode, and for every other ancillary, the legacy time_axis_type
path is unchanged.

The new API makes the file object own the time-varying read: it reads records,
keeps a time window and interpolates into the model fields. This replaces the
time_axis_type list and update_variable_fields for the migrated fields.

Why the changes are needed

  1. The file list is built and the I/O context is closed inside init_io, which
    runs before create_model_data. The fields a file object reads, and the
    collection holding them, must therefore exist before init_io.
  2. The new field objects look for a file variable with the same name as the
    model field. The aerosol file uses different names (e.g. AcSoBC_109 for
    acc_sol_bc), so a mapping is needed.
  3. The file object defaults to non-cyclic, which overrides the cyclic="true"
    in the XML. The aerosol file is a 12-month climatology, so it must be cyclic.
  4. The 17 aerosol fields are also created by create_physics_prognostics.
    Creating them earlier made that creation fail with "already exists".
  5. init_aerosol_fields_alg sets the climatology aerosol fields to zero during
    initialise_model_data, after the file object has already filled them. The
    next refresh is a day later, so the fields stayed zero.

Changes by file

lfric_core

components/lfric-xios/source/lfric_xios_file_mod.f90

  • Adds an optional argument file_variable_names(2,n) to
    lfric_xios_file_constructor. Each column is a pair: the model field name and
    the name of its variable in the file.
  • After a field object is built from fields_in_file, its XIOS name is set from
    the matching pair. Callers that do not pass the argument behave as before.
  • Why: allows model fields to be read from files whose variable names differ
    from the model names.

Currently the migration changes are captured in the branch: https://github.com/ricky-lv426/lfric_core/tree/read_ancils_in_time_windows

lfric_apps

science/gungho/source/driver/init_ancils_mod.f90

  • Adds the public table aerosol_ancil_names(2,17): model field name and the
    corresponding variable name in the glomap aerosol file.
  • Adds create_file_owned_ancils(depository, aerosol_ancil_fields, mesh).
    In updating mode with an aerosol climatology it creates the 17 aerosol fields
    in the depository, sets their write and read behaviour, and adds references to
    the collection given to the file object. It does nothing otherwise.
  • Adds the helper create_file_owned_field. It sets read_field_generic on the
    field at creation because the file object copies the read method into its
    read buffers when the context closes.
  • create_fd_ancils: the aerosol block is a loop over aerosol_ancil_names.
    • Updating mode: setup_ancil_field(..., read_by_file=.true.), so the fields
      are skipped by the generic read and updated by the file object.
    • Fixed mode: the original aerosol_time_axis path, including the coarse mesh
      arguments and the ancil_times_list insert.
  • setup_ancil_field: new optional argument read_by_file. When true, the
    field's read behaviour is removed after creation.

science/gungho/source/driver/gungho_setup_io_mod.F90

  • init_gungho_files gets the aerosol_ancil_fields collection from modeldb
    in updating mode and stops with an error if it is missing.
  • In updating mode the aerosols_ancil file is created with
    OPERATION_TIMESERIES, cyclic=.true., update_freq=1*lx_day,
    file_variable_names=aerosol_ancil_names and fields_in_file. Otherwise it is
    the plain read file as before.
  • The SST file entry is unchanged.

science/gungho/source/driver/gungho_model_mod.F90

  • In initialise_infrastructure, before init_io, creates the empty collection
    aerosol_ancil_fields. With UM physics enabled it also calls
    create_file_owned_ancils.
  • Why: the collection and fields must exist before the file list is built.

science/gungho/source/driver/field_maker_mod.F90

  • add_real_field: if a field of the requested name already exists in the
    depository it is reused. The field is not initialised or added again, but the
    checkpoint and write behaviour and the collection references are applied as
    usual.
  • Why: the aerosol fields are now created before create_physics_prognostics.
    Previously this case stopped with "Field already exists".
  • Fields that do not already exist follow the original path.

interfaces/physics_schemes_interface/source/algorithm/init_aerosol_fields_alg_mod.x90

  • In the GLOMAP climatology branch the 17 climatology aerosol fields are only
    set to zero when ancil_option is not updating. In updating mode the
    ancillary file object owns them.
  • The other fields in that branch (dust, CDNC, diagnostic particle properties)
    are initialised as before.
  • Why: the zeroing ran after the file object had filled the fields, which
    discarded the data until the next daily refresh.

interfaces/jules_interface/source/algorithm/update_ancils_alg_mod.x90

  • tstar_sea is only fetched from ancil_fields when it exists there.
  • Why: with ancil_option = 'updating' and sst_source = 'start_dump'
    tstar_sea is not an ancillary field, and the unconditional get_field
    stopped the run after the first timestep. This is independent of the
    aerosol work.

XML requirements

No XML is changed in this PR currently. To run the aerosol file through the new API in
updating mode, the aerosols_ancil file definition needs:

  • output_freq="1mo" and cyclic="true" on the file;
  • operation="instant" on its fields, or no field entries at all.

The legacy entries in that field group (aerosols_time and the *_data fields)
must be removed for the new API, because the file object enables the whole group
and those definitions use a 12-entry time axis. The legacy path in fixed mode
still needs them, so the two modes currently need different XML.

Status

  • Compiled and run on the HPC (Isambard3). Fixed mode matches main.
  • Updating mode, 120 timesteps with sst_source = 'start_dump': matches main,
    so only the aerosol path differs. Longer runs are still to be checked.
  • Not done: SST and the other ancillaries, regridding for coarse aerosol and
    ozone meshes, a fixed-mode option in core, and the XML changes above.

@github-actions

Copy link
Copy Markdown

⚠️ Hello Ricky Olivier (@ricky-lv426)!

Your CLA signature was found on the base branch, but you appear to have modified the CONTRIBUTORS.md file in this PR.

Please do not edit the CONTRIBUTORS.md file. If you have already signed the CLA, revert changes to the file and your signature will be picked up.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-modified The CLA has been modified as part of this PR - added by GA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants