Github rose-stem and dependencies.yaml

source/Development/TestSuites/jules.rst

@@ -1,11 +1,12 @@
 Testing JULES
 =============
 
-JULES testing is run with the following command from a working copy:
+JULES testing is run with the following command from the top directory of a
+local clone:
 
 .. code-block::
 
-    rose stem --group=all --new
+    cylc vip -z g=all -n <name/of/suite> ./rose-stem
 
 -----
 
@@ -28,6 +29,7 @@ stored :ref:`KGO files <kgo>`.
     See :ref:`multirepo` for details on how to carry out this testing.
 
     .. important::
+
         For **jules-shared** changes, when LFRic testing, the
         changes need to be manually synced to the LFRic location. When UM
         testing, this is not required as **jules-shared** is imported from the
@@ -36,26 +38,26 @@ stored :ref:`KGO files <kgo>`.
 Below is a (by no means comprehensive) set of groups that you may wish to use
 on Met Office systems.
 
-+--------------------+----------------------------------------------------------+
-| Group              | Description                                              |
-+====================+==========================================================+
-+--------------------+----------------------------------------------------------+
-| all                | The complete test suite. This is run automatically       |
-|                    | every night and monitored by the SSD team. All           |
-|                    | :ref:`KGO <kgo>` changing tickets need to run this group.|
-+--------------------+----------------------------------------------------------+
-+--------------------+----------------------------------------------------------+
-| loobos             | A set of tests to exercise these science areas.          |
-|                    |                                                          |
-| gswp2              |                                                          |
-|                    |                                                          |
-| eraint             |                                                          |
-|                    |                                                          |
-| imogen             |                                                          |
-+--------------------+----------------------------------------------------------+
-| xc40/linux         | All tests designed to run on the named platform.         |
-+--------------------+----------------------------------------------------------+
-| scripts            | All of the auxillary scripts that are designed to check  |
-|                    | the code standards in ways that aren't tested by the     |
-|                    | compiler.                                                |
-+--------------------+----------------------------------------------------------+
++----------------------+----------------------------------------------------------+
+| Group                | Description                                              |
++======================+==========================================================+
++----------------------+----------------------------------------------------------+
+| all                  | The complete test suite. This is run automatically       |
+|                      | every night and monitored by the SSD team. All           |
+|                      | :ref:`KGO <kgo>` changing tickets need to run this group.|
++----------------------+----------------------------------------------------------+
++----------------------+----------------------------------------------------------+
+| loobos               | A set of tests to exercise these science areas.          |
+|                      |                                                          |
+| gswp2                |                                                          |
+|                      |                                                          |
+| eraint               |                                                          |
+|                      |                                                          |
+| imogen               |                                                          |
++----------------------+----------------------------------------------------------+
+| ex1a/azspice         | All tests designed to run on the named platform.         |
++----------------------+----------------------------------------------------------+
+| scripts              | All of the auxillary scripts that are designed to check  |
+|                      | the code standards in ways that aren't tested by the     |
+|                      | compiler.                                                |
++----------------------+----------------------------------------------------------+

source/Development/TestSuites/lfric_apps.rst

@@ -5,15 +5,11 @@ Testing LFRic Apps
 
 Rose stem:
 
-    LFRic Apps testing uses rose-stem and is run with the following commands
-    from a working copy:
+    The LFRic Apps rose-stem test suite can be run by,
 
     .. code-block:: shell
 
-        export CYLC_VERSION=8
-        rose stem --group=developer
-        cylc play <working copy name>
-        cylc gui
+        cylc vip -z g=developer -n <name/of/suite> ./rose-stem
 
 Local testing:
 
@@ -24,21 +20,6 @@ Local testing:
     This test does not use rose or cylc and is particularly useful for checking
     for compile errors while developing.
 
------
-
-.. important::
-
-    When specifying the lfric_core source the lfric_core revision **must** be
-    updated in ``dependencies.sh``.
-
-    * If setting the source to an fcm URL, the mirror (``.xm_``) needs to be
-      used and the revision can either be blank (for latest commit) or any
-      valid revision for that branch.
-    * If setting the source to a Working Copy, the hostname needs to be
-      provided (as Hostname:Path) and the revision must be blank.
-
-    For more details, see :ref:`multi-repo_testing`.
-
 
 Rose stem
 ---------
@@ -56,7 +37,7 @@ and plot tasks to confirm the outputs.
 Below is a (by no means comprehensive) set of groups that you may wish to use
 on Met Office systems. Note that there is a lot of overlap between these
 groups, and that you can specify more than one at once, e.g.
-``--group=developer,gungho_model``.
+``-z g=developer,gungho_model``.
 
 +--------------------+----------------------------------------------------------+
 | Group              | Description                                              |
@@ -77,7 +58,7 @@ groups, and that you can specify more than one at once, e.g.
 +--------------------+----------------------------------------------------------+
 | integration_tests  | Integration tests for all applications and science areas |
 +--------------------+----------------------------------------------------------+
-| xc40/spice/monsoon | All tests designed to run on the named platform.         |
+| ex1a/azspice       | All tests designed to run on the named platform.         |
 +--------------------+----------------------------------------------------------+
 | scripts            | All of the auxillary scripts that are designed to check  |
 |                    | the code standards in ways that aren't tested by the     |

source/Development/TestSuites/lfric_core.rst

@@ -9,14 +9,11 @@ Testing LFRic Core
     and the make test-suite functionality removed. Only the cylc8 suite is now
     maintained.
 
-LFRic testing is launched with Cylc8 rose-stem commands (as in eg. LFRic
-Apps):
+LFRic testing can be launched from the top directory of a local clone,
 
 .. code-block:: shell
 
-    export CYLC_VERSION=8
-    rose stem --group=developer
-    cylc play <NAME-OF-SUITE>
+    cylc vip -z g=developer -n <name/of/suite> ./rose-stem
 
 -----
 
@@ -26,14 +23,14 @@ launched from make and utilise rose and cylc.
 
 While developing your change, for expediency you may want to run the tests for
 only some applications. This can be done by changing the group you run, eg
-``--group=simple_diffusion``.
+``-z g=simple_diffusion``.
 
 The command above will launch the developer suite. You can include slightly
-more testing if required by running ``--group=all`` instead (this includes the
+more testing if required by running ``-z g=all`` instead (this includes the
 developer suite).
 
-It is also possible to run on a single platform, eg. ``--group=ex1a``. To
-select which meto EX machine is used, add ``-S USE_EX<AB/CD/Z>``.
+It is also possible to run on a single platform, eg. ``-z g=ex1a``. To
+select which meto EX machine is used, add ``-S USE_EX<AB/CD/Z>=true``.
 
 .. tip::
 

source/Development/TestSuites/multi-repo_testing.rst

@@ -8,96 +8,84 @@ repositories involved. To carry out the tests involved in a linked ticket it
 can be helpful to refer to the :ref:`repository figure <multirepo>`; testing
 both child and parent repositories as needed. Further details of how testing
 in each repository is handled can be found on the :ref:`Testing
-page<testing>`. Compatible code revisions are needed for testing across
-repositories as described above.
+page<testing>`.
+
+All Simulation Systems repositories containing a test suite will also contain a
+``dependencies.yaml`` file in the top directory of the repository. This file
+contains the details of all sources used by the test suite in the format (using
+lfric_core as an example):
+
+.. code-block:: yaml
+
+    lfric_core:
+        source: git@github.com:MetOffice/lfric_core.git
+        ref: <Long Hash / Tag>
+
+The ``source`` setting sets the location of the repository on github. By
+default, the test suite will access github repositories by using ssh, as shown
+by the ``git@github.com`` part of the source. This line can be modified to point
+at users fork of the repository instead, eg.
+``source: git@github.com:UserName/lfric_core.git``. The ``source`` can also be a
+local git clone, in which case it should take the form,
+``source: <HOSTNAME>:/path/to/clone``.
+
+The ``ref`` setting takes a git tree-ish value. Common settings will be a commit
+hash, a tag or a branch name. At release, they will be tags, but as linked PRs
+are committed, they will be changed to the long form of the relevant commit
+hash (although using a short version of the hash will also work). If testing a
+linked branch, then you can either set to the name of the branch, or a commit
+associated with that branch. It is also recommended to set this, even when
+setting the source to a local clone. That way if you switch branches in the
+clone, the correct branch for testing will be used. Finally, if left blank, the
+Head of the repositories default branch will be used if a github source. If it
+is a local clone, ref is determined by the state of the repository at source
+extraction time.
+
+Various different configurations of an lfric_core source are shown below with an
+explanation of each,
+
+.. code-block:: yaml
+
+    # The upstream lfric_core repository at tag 3.0
+    lfric_core:
+        source: git@github.com:MetOffice/lfric_core.git
+        ref: core3.0
+
+    # The upstream lfric_core repository at a specific commit hash
+    lfric_core:
+        source: git@github.com:MetOffice/lfric_core.git
+        ref: a1b2c3d4e5f67890abcdef1234567890abcdef12
+
+    # As above, but with a shortened form of the hash (7 characters in this case)
+    lfric_core:
+        source: git@github.com:MetOffice/lfric_core.git
+        ref: a1b2c3d
+
+    # The upstream lfric_core repository, with the default branch (main) at Head
+    lfric_core:
+        source: git@github.com:MetOffice/lfric_core.git
+        ref:
+
+    # A Users fork of the lfric_core repoistory, on branch my_branch
+    lfric_core:
+        source: git@github.com:UserName/lfric_core.git
+        ref: my_branch
+
+    # A Users fork of the lfric_core repository at a specific hash
+    lfric_core:
+        source: git@github.com:UserName/lfric_core.git
+        ref: f9e8d7c
+
+    # A local clone of the lfric_core repository, pointing at my_branch
+    lfric_core:
+        source: hostname:/path/to/lfric_core
+        ref: my_branch
+
+    # A local clone of the lfric_core repository, pointing at a specific hash
+    lfric_core:
+        source: hostname:/path/to/lfric_core
+        ref: f9e8d7c
 
-Testing changes in JULES, LFRic Core, UKCA, or any other child repositories is
-as simple as running the standalone test procedures for these codebases.
-
-.. important::
-
-    When specifying an alternative source in the ``dependencies.sh`` file the
-    revision for the source **must** be updated.
-
-    * If setting the source as an fcm URL, the mirror (``.xm_``) needs to be
-      used and the revision can either be blank (for latest commit) or any
-      valid revision for that branch.
-    * If setting the source as a Working Copy, the hostname needs to be
-      provided (as Hostname:Path) and the revision must be blank.
-
-Testing the UM with other repositories
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To test the UM, any changes to JULES, UKCA, Socrates, CASIM etc will also need
-to be included. This is done by adding another source to the rose stem command
-line.
-
-1. Checkout a UM working copy
-    - this may be your branch from a linked ticket, or a clean trunk copy at
-      either the last release or a suitable head of trunk revision.
-
-2. Run rose stem, including a source code path to every branch involved. As a
-   minimum run ``developer`` group and all groups that cover the repositories
-   being tested.
-
-.. code-block:: shell
-
-    rose stem --group=developer,jules,ukca --source=. \
-        --source=/path/to/jules/changes --source=/path/to/ukca/changes
-
-The source paths involved can either be to local working copies or links to the
-fcm source control e.g. ``fcm:jules.xm_br/dev/user/branch_name``. As many
-source paths as needed can be added to the list.
-
-Testing LFRic Apps with other repositories
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-LFRic Apps testing needs to encompass all of the other repositories affected.
-Paths to the other codebases involved should be added to ``dependencies.sh``
-under each of the ``*_sources`` variables. Again these paths can either be to
-local changes or those in the repository.
-
-1. Checkout an LFRic Apps working copy
-
-- this may be your branch from a linked ticket, or a clean trunk copy at either
-  the last release or a suitable head of trunk revision.
-
-2. Update dependencies.sh to point to all other code changes, e.g.
-
-.. code-block::
-
-    lfric_core_rev=
-    lfric_core_sources=fcm:lfric.xm_br/path/to/branch
-
-    casim_rev=
-    casim_sources=vldXXX:/path/to/casim/working/copy
-
-3a. Run the lfric_atm developer test-suite
-
-- suitable for testing changes in other repositories that do not include any
-  LFRic Apps changes
-
-.. code-block:: shell
-
-    export CYLC_VERSION=8
-    rose stem --group=lfric_atm_developer
-    cylc play <working copy name>
-    cylc gui
-
-3b. Run the full developer test-suite
-
-- suitable for testing LFRic Apps changes with other repositories, or expanding
-  testing if lfric_atm tests have shown errors.
-
-.. code-block:: shell
-
-    export CYLC_VERSION=8
-    rose stem --group=developer
-    cylc play <working copy name>
-    cylc gui
-
-More details on LFRic Apps testing are found on the
-:ref:`Testing LFRic Apps page<lfric_apps_test>`.
 
 .. note::
 

source/Development/TestSuites/ukca.rst

@@ -8,3 +8,10 @@ with both the UM and LFRic by following the :ref:`linked tickets guidance
 For further guidance on testing and working with UKCA, including standard
 suites and box models see the `UKCA trac wiki
 <https://code.metoffice.gov.uk/trac/ukca/wiki/WorkingPractices>`__.
+
+There also exists a small UKCA rose-stem suite, which contains a code styling
+check. This can launched from the top directory of a working copy by running,
+
+.. code-block::
+
+    cylc vip -z g=scripts -n <name/of/suite> ./rose-stem