Skip to content

Github rose-stem and dependencies.yaml #443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jenny Hickson (jennyhickson) merged 17 commits into from
Sep 23, 2025
Merged

Github rose-stem and dependencies.yaml #443

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d57a58d
in progress test suite
james-bruten-mo Sep 19, 2025
4bcdfe5
review rose-stem changes
james-bruten-mo Sep 22, 2025
72aa6b7
Merge branch 'github_wps' into gh_test_suites
james-bruten-mo Sep 22, 2025
454fe41
add -S options
james-bruten-mo Sep 22, 2025
18f5060
revert changes to macros section
james-bruten-mo Sep 23, 2025
ebf54f8
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
8386fcf
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
9dd02ce
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
91d1c36
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
b0ad6d2
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
0e825eb
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
8f9003c
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
5a442fd
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
904a713
Apply suggestion from @jennyhickson
james-bruten-mo Sep 23, 2025
5c6525e
-z group=
james-bruten-mo Sep 23, 2025
095a4b2
move around new cylc vip
james-bruten-mo Sep 23, 2025
96609d8
whitespace
james-bruten-mo Sep 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 27 additions & 25 deletions source/Development/TestSuites/jules.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
Testing JULES
=============

JULES testing is run with the following command from a working copy:
JULES testing is run with the following command from the top directory of a
local clone:

.. code-block::

rose stem --group=all --new
cylc vip -z group=all -n <name/of/suite> ./rose-stem

-----

Expand All @@ -28,6 +29,7 @@ stored :ref:`KGO files <kgo>`.
See :ref:`multirepo` for details on how to carry out this testing.

.. important::

For **jules-shared** changes, when LFRic testing, the
changes need to be manually synced to the LFRic location. When UM
testing, this is not required as **jules-shared** is imported from the
Expand All @@ -36,26 +38,26 @@ stored :ref:`KGO files <kgo>`.
Below is a (by no means comprehensive) set of groups that you may wish to use
on Met Office systems.

+--------------------+----------------------------------------------------------+
| Group | Description |
+====================+==========================================================+
+--------------------+----------------------------------------------------------+
| all | The complete test suite. This is run automatically |
| | every night and monitored by the SSD team. All |
| | :ref:`KGO <kgo>` changing tickets need to run this group.|
+--------------------+----------------------------------------------------------+
+--------------------+----------------------------------------------------------+
| loobos | A set of tests to exercise these science areas. |
| | |
| gswp2 | |
| | |
| eraint | |
| | |
| imogen | |
+--------------------+----------------------------------------------------------+
| xc40/linux | All tests designed to run on the named platform. |
+--------------------+----------------------------------------------------------+
| scripts | All of the auxillary scripts that are designed to check |
| | the code standards in ways that aren't tested by the |
| | compiler. |
+--------------------+----------------------------------------------------------+
+----------------------+----------------------------------------------------------+
| Group | Description |
+======================+==========================================================+
+----------------------+----------------------------------------------------------+
| all | The complete test suite. This is run automatically |
| | every night and monitored by the SSD team. All |
| | :ref:`KGO <kgo>` changing tickets need to run this group.|
+----------------------+----------------------------------------------------------+
+----------------------+----------------------------------------------------------+
| loobos | A set of tests to exercise these science areas. |
| | |
| gswp2 | |
| | |
| eraint | |
| | |
| imogen | |
+----------------------+----------------------------------------------------------+
| ex1a/azspice | All tests designed to run on the named platform. |
+----------------------+----------------------------------------------------------+
| scripts | All of the auxillary scripts that are designed to check |
| | the code standards in ways that aren't tested by the |
| | compiler. |
+----------------------+----------------------------------------------------------+
28 changes: 5 additions & 23 deletions source/Development/TestSuites/lfric_apps.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,12 @@ Testing LFRic Apps

Rose stem:

LFRic Apps testing uses rose-stem and is run with the following commands
from a working copy:
The LFRic Apps rose-stem test suite can be run from the top directory of a
local clone:

.. code-block:: shell

export CYLC_VERSION=8
rose stem --group=developer
cylc play <working copy name>
cylc gui
cylc vip -z group=developer -n <name/of/suite> ./rose-stem

Local testing:

Expand All @@ -24,21 +21,6 @@ Local testing:
This test does not use rose or cylc and is particularly useful for checking
for compile errors while developing.

-----

.. important::

When specifying the lfric_core source the lfric_core revision **must** be
updated in ``dependencies.sh``.

* If setting the source to an fcm URL, the mirror (``.xm_``) needs to be
used and the revision can either be blank (for latest commit) or any
valid revision for that branch.
* If setting the source to a Working Copy, the hostname needs to be
provided (as Hostname:Path) and the revision must be blank.

For more details, see :ref:`multi-repo_testing`.


Rose stem
---------
Expand All @@ -56,7 +38,7 @@ and plot tasks to confirm the outputs.
Below is a (by no means comprehensive) set of groups that you may wish to use
on Met Office systems. Note that there is a lot of overlap between these
groups, and that you can specify more than one at once, e.g.
``--group=developer,gungho_model``.
``-z group=developer,gungho_model``.

+--------------------+----------------------------------------------------------+
| Group | Description |
Expand All @@ -77,7 +59,7 @@ groups, and that you can specify more than one at once, e.g.
+--------------------+----------------------------------------------------------+
| integration_tests | Integration tests for all applications and science areas |
+--------------------+----------------------------------------------------------+
| xc40/spice/monsoon | All tests designed to run on the named platform. |
| ex1a/azspice | All tests designed to run on the named platform. |
+--------------------+----------------------------------------------------------+
| scripts | All of the auxillary scripts that are designed to check |
| | the code standards in ways that aren't tested by the |
Expand Down
15 changes: 6 additions & 9 deletions source/Development/TestSuites/lfric_core.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,11 @@ Testing LFRic Core
and the make test-suite functionality removed. Only the cylc8 suite is now
maintained.

LFRic testing is launched with Cylc8 rose-stem commands (as in eg. LFRic
Apps):
LFRic testing can be launched from the top directory of a local clone,

.. code-block:: shell

export CYLC_VERSION=8
rose stem --group=developer
cylc play <NAME-OF-SUITE>
cylc vip -z group=developer -n <name/of/suite> ./rose-stem

-----

Expand All @@ -26,14 +23,14 @@ launched from make and utilise rose and cylc.

While developing your change, for expediency you may want to run the tests for
only some applications. This can be done by changing the group you run, eg
``--group=simple_diffusion``.
``-z group=simple_diffusion``.

The command above will launch the developer suite. You can include slightly
more testing if required by running ``--group=all`` instead (this includes the
more testing if required by running ``-z group=all`` instead (this includes the
developer suite).

It is also possible to run on a single platform, eg. ``--group=ex1a``. To
select which meto EX machine is used, add ``-S USE_EX<AB/CD/Z>``.
It is also possible to run on a single platform, eg. ``-z group=ex1a``. To
select which meto EX machine is used, add ``-S USE_EX<AB/CD/Z>=true``.

.. tip::

Expand Down
166 changes: 77 additions & 89 deletions source/Development/TestSuites/multi-repo_testing.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,96 +8,84 @@ repositories involved. To carry out the tests involved in a linked ticket it
can be helpful to refer to the :ref:`repository figure <multirepo>`; testing
both child and parent repositories as needed. Further details of how testing
in each repository is handled can be found on the :ref:`Testing
page<testing>`. Compatible code revisions are needed for testing across
repositories as described above.
page<testing>`.

All Simulation Systems repositories containing a test suite will also contain a

@jennyhickson Jenny Hickson (jennyhickson) Sep 23, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than just UM and LFRic?

@james-bruten-mo James Bruten (james-bruten-mo) Sep 23, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, discussed in person

``dependencies.yaml`` file in the top directory of the repository. This file
contains the details of all sources used by the test suite in the format (using
lfric_core as an example):

.. code-block:: yaml

lfric_core:
source: git@github.com:MetOffice/lfric_core.git
ref: <Long Hash / Tag>

The ``source`` setting sets the location of the repository on github. By
default, the test suite will access github repositories by using ssh, as shown
by the ``git@github.com`` part of the source. This line can be modified to point
at users fork of the repository instead, eg.
``source: git@github.com:UserName/lfric_core.git``. The ``source`` can also be a
local git clone, in which case it should take the form,
``source: <HOSTNAME>:/path/to/clone``.

The ``ref`` setting takes a git tree-ish value. Common settings will be a commit
hash, a tag or a branch name as demostrated by the examples below. At
release, the refs will be tags and will be changed to the long form of the relevant
commit hash as part of linked pull requests.

If left blank the behaviour depends on the source:

* **a github source:** the Head of the repositories default branch will be used.
* **a local clone:** the state of the repository at source extraction time will be used.
It is recommened to set a ref when setting the source to a local clone. That way
if you switch branches in the clone, the correct branch for testing will be used.

Various different configurations of an lfric_core source are shown below with an
explanation of each,

.. code-block:: yaml

# The upstream lfric_core repository at tag 3.0
lfric_core:
source: git@github.com:MetOffice/lfric_core.git
ref: core3.0

# The upstream lfric_core repository at a specific commit hash
lfric_core:
source: git@github.com:MetOffice/lfric_core.git
ref: a1b2c3d4e5f67890abcdef1234567890abcdef12

# As above, but with a shortened form of the hash (7 characters in this case)
lfric_core:
source: git@github.com:MetOffice/lfric_core.git
ref: a1b2c3d

# The upstream lfric_core repository, with the default branch (main) at Head
lfric_core:
source: git@github.com:MetOffice/lfric_core.git
ref:

# A Users fork of the lfric_core repoistory, on branch my_branch
lfric_core:
source: git@github.com:UserName/lfric_core.git
ref: my_branch

# A Users fork of the lfric_core repository at a specific hash
lfric_core:
source: git@github.com:UserName/lfric_core.git
ref: f9e8d7c

# A local clone of the lfric_core repository, pointing at my_branch
lfric_core:
source: hostname:/path/to/lfric_core
ref: my_branch

# A local clone of the lfric_core repository, pointing at a specific hash
lfric_core:
source: hostname:/path/to/lfric_core
ref: f9e8d7c

Testing changes in JULES, LFRic Core, UKCA, or any other child repositories is
as simple as running the standalone test procedures for these codebases.

.. important::

When specifying an alternative source in the ``dependencies.sh`` file the
revision for the source **must** be updated.

* If setting the source as an fcm URL, the mirror (``.xm_``) needs to be
used and the revision can either be blank (for latest commit) or any
valid revision for that branch.
* If setting the source as a Working Copy, the hostname needs to be
provided (as Hostname:Path) and the revision must be blank.

Testing the UM with other repositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To test the UM, any changes to JULES, UKCA, Socrates, CASIM etc will also need
to be included. This is done by adding another source to the rose stem command
line.

1. Checkout a UM working copy
- this may be your branch from a linked ticket, or a clean trunk copy at
either the last release or a suitable head of trunk revision.

2. Run rose stem, including a source code path to every branch involved. As a
minimum run ``developer`` group and all groups that cover the repositories
being tested.

.. code-block:: shell

rose stem --group=developer,jules,ukca --source=. \
--source=/path/to/jules/changes --source=/path/to/ukca/changes

The source paths involved can either be to local working copies or links to the
fcm source control e.g. ``fcm:jules.xm_br/dev/user/branch_name``. As many
source paths as needed can be added to the list.

Testing LFRic Apps with other repositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

LFRic Apps testing needs to encompass all of the other repositories affected.
Paths to the other codebases involved should be added to ``dependencies.sh``
under each of the ``*_sources`` variables. Again these paths can either be to
local changes or those in the repository.

1. Checkout an LFRic Apps working copy

- this may be your branch from a linked ticket, or a clean trunk copy at either
the last release or a suitable head of trunk revision.

2. Update dependencies.sh to point to all other code changes, e.g.

.. code-block::

lfric_core_rev=
lfric_core_sources=fcm:lfric.xm_br/path/to/branch

casim_rev=
casim_sources=vldXXX:/path/to/casim/working/copy

3a. Run the lfric_atm developer test-suite

- suitable for testing changes in other repositories that do not include any
LFRic Apps changes

.. code-block:: shell

export CYLC_VERSION=8
rose stem --group=lfric_atm_developer
cylc play <working copy name>
cylc gui

3b. Run the full developer test-suite

- suitable for testing LFRic Apps changes with other repositories, or expanding
testing if lfric_atm tests have shown errors.

.. code-block:: shell

export CYLC_VERSION=8
rose stem --group=developer
cylc play <working copy name>
cylc gui

More details on LFRic Apps testing are found on the
:ref:`Testing LFRic Apps page<lfric_apps_test>`.

.. note::

Expand Down
7 changes: 7 additions & 0 deletions source/Development/TestSuites/ukca.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,10 @@ with both the UM and LFRic by following the :ref:`linked tickets guidance
For further guidance on testing and working with UKCA, including standard
suites and box models see the `UKCA trac wiki
<https://code.metoffice.gov.uk/trac/ukca/wiki/WorkingPractices>`__.

There also exists a small UKCA rose-stem suite, which contains a code styling
check. This can launched from the top directory of a local clone by running,

.. code-block::

cylc vip -z group=scripts -n <name/of/suite> ./rose-stem
Loading