Merge in changes from main

source/Development/documentation.rst

@@ -14,6 +14,7 @@ notably:
 | LFRic Documentation Papers |`view LFRic`_                | `edit LFRIc`_                                   |
 +----------------------------+-----------------------------+-------------------------------------------------+
 
+
 LFRic Apps and Core also use doxygen to document the code and all changes
 should include appropriate doxygen changes to go with them. Doxygen guidelines
 are available on the `LFRic Technical pages

source/Development/inputs.rst

@@ -90,13 +90,17 @@ whether an upgrade macro is necessary.
 
     rose edit -C rose-stem/app/APP-NAME
 
+  Note for MetOffice Users: ``rose edit`` runs on a server, so make sure your
+  source can be seen from a server - i.e. not on a drive that's only available
+  to your local machine.
+
   For LFRic Apps a few extra changes are required. In your branch (your test
   branch if you have an upgrade macro):
 
   .. code-block:: shell
 
     cd rose-meta
-    rose edit -C rose-stem/app/APP-NAME --no-warn version
+    rose edit -C ../rose-stem/app/APP-NAME --no-warn version
 
   If you have a linked LFRic Core or Jules ticket with metadata changes, you
   can load their metadata by adding ``-M /path/to/working_copy/rose-meta`` to

source/Reviewers/curaterelease.rst

@@ -12,6 +12,12 @@ Curating a Release
     releases/mule_release
     releases/um_main_release
     releases/lfric_apps_release
+    releases/release_notes
+    releases/standard_suites
+    releases/stash_browser
+    releases/umdp_release
+    releases/wiki_pages
+    releases/shared_accounts
     releases/updating_prebuilds
 
 .. _reference-tagging:
@@ -67,12 +73,11 @@ Open a UM X.Y release Curation Ticket, and assign tasks as a team,
     ||  || LFRic Apps Release                                                                         ||  ||
     ||||||'''Post Release'''||
     ||  || Release notes                                                                              ||  ||
-    ||  || Update standard suites & finalise std jobs page                                            ||  ||
-    ||  || Update $UMDIR scripts                                                                      ||  ||
+    ||  || Update standard suites                                                                     ||  ||
     ||  || Check resource monitoring scripts still work                                               ||  ||
-    ||  || Install Code and Stash browsers                                                            ||  ||
+    ||  || Install the Stash Browser                                                                  ||  ||
     ||  || UMDP3 Release                                                                              ||  ||
-    ||  || Update wikis, working practices, and create bit comp table                                 ||  ||
+    ||  || Standard Jobs + Wiki Page                                                                  ||  ||
     ||  || Review and update trunk and shared account permissions                                     ||  ||
 
     [https://metoffice.github.io/simulation-systems/Reviewers/curaterelease.html Curating a release Page]
@@ -152,51 +157,44 @@ All LFRic Tickets (Apps + Core), Jules Release
 Post Release Tasks
 ------------------
 
-`Release Notes <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#ReleaseNotes>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+`Release Notes <release_notes>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
 Most of this can be done pre-release but some details of revision numbers will
 be dependent on the main release being done.
 
 
-`Upgrading Standalone Suites <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#UpgradingStandaloneSuites>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+:ref:`Updating Standard Suites <standard_suites>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
-UM Release (for UM suites), Apps Release (for Apps suites)
+UM + Apps Releases
 
 
-`Standard Jobs Page <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#FinalizeTheStandardJobspage>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+`Stash Browser <stash_browser>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
 UM Release
 
 
-`Code and Stash Browsers <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#InstallCodeandStashbrowsers>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-**Dependencies**
-UM Release
-
-
-`UMDP Release <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#UMDPrelease>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+`UMDP Release <umdp_release>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
 UM Release, Standard Suites Upgrade
 
 
-`Wikis Update <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#Updatewikisworkingpracticesandcreatebitcomptable>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+:ref:`Standard Jobs and Wiki Pages <wiki_pages>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
-UM Release
+UM + Apps Releases
 
 
-`Shared Account Permissions <https://code.metoffice.gov.uk/trac/um/wiki/CuratingARelease#Reviewandupdatetrunkandsharedaccountpermissions>`_
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+:ref:`Trunk and Shared Account Permissions <shared_accounts>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 **Dependencies**
 None

source/Reviewers/releases/lfric_apps_release.rst

@@ -9,8 +9,8 @@ LFRic Inputs KGO Install
   have been committed.
 * It's easiest to use the umtest nightly testing for this and will save having
   to run the suite twice.
-* Alternatively, run ``rose stem --group=lfricinputs`` and wait for this to
-  finish - all jobs should pass.
+* Alternatively, run ``rose stem --group=lfricinputs -S HOUSEKEEPING=false``
+  and wait for this to finish - all jobs should pass.
 * Install the kgo by running
   ``$UMDIR/SimSys_Scripts/kgo_updates/meto_update_kgo.sh --new-release``
 
@@ -46,15 +46,25 @@ LFRic Release
     * Add a ``version_ab_xy.py`` upgrade file - a copy of the versions.py file
     * Reset the ``versions.py`` file with no upgrade macros
 
+* Tag other repositories and update dependencies.sh:
+
+  * :ref:`Tag <reference-tagging>` CASIM, JULES, SOCRATES and UKCA with ``appsX.Y=revision``
+  * In dependencies.sh:
+
+    * Make sure ``lfric_core_sources`` is pointing at the core working copy with ``lfric_core_rev`` blank
+    * Update ``*_rev`` for all other repositories to be ``appsX.Y`` with ``*_sources`` blank
+
+* Commit your changes to both Apps and Core branches.
+
 * Run the test suites
 
     * ``rose stem --group=all`` for both Apps and Core.
     * Make sure ``dependencies.sh`` is pointing at the core working copy
 
-* Once testing is complete, reset the ``dependencies.sh``
+* Once testing is complete, update LFRic Core in ``dependencies.sh``
 
-    * LFRic Core should be ``coreX.Y``
-    * All others should be ``umC.D``
+  * ``lfric_core_rev`` should be ``coreX.Y``
+  * ``lfric_core_sources`` should be blank
 
 * Get the tickets reviewed and committed:
 

source/Reviewers/releases/partner_testing.rst

@@ -24,3 +24,7 @@ scope changes, eg. to just the Met Office site, can still go on.
     messages to ensure that all international partners receive the email.
     Change this setting if you receive bounced emails during partner testing.
 
+
+.. important::
+
+    Now is also a good time to ask for the Jules release notes. This is usually done by a member of the Jules community, so email them advising them of the likely release date. Ask the team for their details if unsure.

source/Reviewers/releases/release_notes.rst

@@ -0,0 +1,82 @@
+.. _release_notes:
+
+Release Notes
+=============
+
+The below shows the trac release notes template previously used. This will want editing and the exact location needs deciding for the move to github. However it can be used as an initial guide to the sort of information included.
+
+.. code-block::
+
+    [[PageOutline]]
+    -----
+    = UM vnXX.Y User Release Notes =
+
+    == Summary ==
+
+    * Release date:
+    * Number of tickets:
+
+    === Science highlights ===
+
+    === Technical highlights ===
+
+    === Optimisation highlights ===
+
+    == Known Issues ==
+
+    None
+
+    == Revisions and Keywords ==
+
+    || Repository || !Keyword/Revision ||
+    || [browser:main UM]                                                    || vnXX.Y = NNNN ||
+    || [https://code.metoffice.gov.uk/trac/jules/browser/main JULES]        || vnA.B = umXX.y = NNNN ||
+    || [https://code.metoffice.gov.uk/trac/monc/browser/casim CASIM]        || etc ||
+    || [https://code.metoffice.gov.uk/trac/socrates/browser/main SOCRATES]  ||     ||
+    || [https://code.metoffice.gov.uk/trac/ukca/browser/main UKCA]          ||     ||
+    || [browser:aux Aux]                                                    ||     ||
+
+    GCOM and SHUMlib installations should be built using revisions:
+
+    || Repository || !Keyword/Revision ||
+    || [https://code.metoffice.gov.uk/trac/gcom/browser/main GCOM]          || vnXX.Y = NNNN ||
+    || [https://code.metoffice.gov.uk/trac/utils/browser/shumlib SHUMlib]   || YYYY.MM.N = NNNN ||
+
+    Note that SHUMlib may not be released at every UM release.
+
+    For testing purposes, the LFRic Apps and MOCI repositories:
+
+    || Repository || Revision ||
+    || [browser:main/trunk/rose-stem/rose-suite.conf MOCI]           ||  NNNN  ||
+    || [https://code.metoffice.gov.uk/trac/lfric_apps/browser/main LFRic Apps] ||  vnA.B = umXX.Y = NNNN ||
+
+    Required for rose-stem:
+
+    {{{#!comment
+    Obtain these by grepping, for example:
+    grep -r CYLC_VERSION ~frzz/cylc-run/um_heads_nightly_YYYY-MM-DD/log/job
+    }}}
+
+    * Rose vn YYYY.MM.N
+    * Cylc vn YYYY.MM.N
+
+    == Ticket Overview ==
+
+    * Tickets marked as closed/fixed below may not be associated with a [log:main/trunk trunk commit]
+    * Only tickets with a UM element are displayed below. Sub-repositories may contain additional standalone tickets that will have been implicitly tested and included in the release.
+
+    [[TicketQuery(milestone^=UMXX.Y (MMM-YY),status=closed,resolution=fixed,format=table,col=summary|reporter|keywords)]]
+
+    == Platforms ==
+
+    Testing:
+    * Met Office SPICE: [wiki:ReleaseNotesXX.Y/ReleaseTestingAZSPICE]
+    * Met Office EX1A: [wiki:ReleaseNotesXX.Y/ReleaseTestingEX1A]
+
+
+For the highlights section:
+
+* Contact HPC Optimisation team (usually AM) to request details for the optimisation section
+* Take some time to think about the 'big picture' relevance of tickets for the science and technical highlights
+* Copy the 'head of trunk' section and paste underneath, changing the section title to the release.
+* Paste trac.logs into the relevant sub-pages. You'll likely be able to use some nightly testing runs.

source/Reviewers/releases/shared_accounts.rst

@@ -0,0 +1,42 @@
+.. _shared_accounts:
+
+Trunk and Shared Accounts Permissions
+=====================================
+
+.. warning::
+
+    Incorrectly editing the svnperms.conf file can break all access to a repo
+
+To update fcm repositories trunk access list (using UM as an example),
+
+* First checkout the top level of the repo,
+
+  .. code-block::
+
+    fcm co --depth=files https://code.metoffice.gov.uk/svn/um
+
+* Move into the working copy and make the required changes in the ``svnperms.conf`` file.
+* Commit the change.
+
+The above process needs doing for all of,
+
+* UM
+* LFRic Apps
+* Jules
+* UKCA
+* Casim
+* Shumlib
+* MOCI
+
+Modify SSD Team Github Permissions:
+
+* https://github.com/orgs/jules-lsm/teams/system-managers
+* ​https://github.com/orgs/MetOffice/teams/ssdteam
+
+Review account permissions in Active Directory for the following Shared Users:
+
+* umadmin
+* umtest
+* julesadmin
+* lfricadmin
+* spackadmin

source/Reviewers/releases/standard_suites.rst

@@ -0,0 +1,96 @@
+.. _standard_suites:
+
+Standard Suites
+===============
+
+UM Standard Suites
+------------------
+
+For the UM the following Suites must be updated:
+
+* UM Azspice - `u-aa003 <https://code.metoffice.gov.uk/trac/roses-u/browser/a/a/0/0/3/trunk>`_
+* UM EX1A - `u-ci752 <https://code.metoffice.gov.uk/trac/roses-u/browser/c/i/7/5/2/trunk>`_
+* SCM - `u-aa741 <https://code.metoffice.gov.uk/trac/roses-u/browser/a/a/7/4/1/trunk>`_
+* CreateBC - `u-aa258 <https://code.metoffice.gov.uk/trac/roses-u/browser/a/a/2/5/8/trunk>`_
+
+First get a copy of the UM meta trunk, ``fcm co fcm:um_meta.x_tr /path/to/meta/trunk``. Similarly, get a copy of Jules at the most recent release. Then upgrade each suite by doing the following,
+
+* Checkout the suite. ``rosie checkout u-xxNNN`` will put a copy in ``~/roses``.
+* Check whether the ``access-list`` in the ``rose-suite.conf`` needs updating - if you are not on it, ask someone to add you now and update your copy once done.
+* Move into the suite and edit the ``rose-suite.conf`` file,
+
+  * If it has a ``VN`` setting, update it to the new version.
+  * If it has a ``prebuild`` path, update that now. If there are prebuilds, check the ``flow.cylc`` for any prebuild path overrides. These exist particularly on the EXs.
+
+* Validate the existing apps by running,
+
+  .. code-block::
+
+    rose macro --fix -C app/APP
+
+  where ``APP`` should be replaced by each available app in turn. It should report that 0 changes were made.
+* Now upgrade the apps to the new version, using the metadata trunk checked out earlier. Again, replace ``APP`` with each app in turn. This will prompt you to confirm while running.
+
+  .. code-block::
+
+    rose app-upgrade -C app/APP -M /path/to/the/meta/trunk vnX.Y
+
+* Finally run the validator macros. If everything is fine the command will report nothing, however the SCM suite will raise some warnings about Openmp and "global rows" - these can be ignored.
+
+  .. code-block::
+
+    rose macro metomi.rose.macros.DefaultValidators -M /path/to/the/meta/trunk:/path/to/the/jules/hot/rose-meta
+
+* Check that the changes look sensible and then run the suite,
+
+  .. code-block::
+
+    fcm diff -g
+    cylc vip
+
+.. _suite_commit:
+
+* Once the suite has succeeded, commit the changes (``fcm ci``) with a suitable message and note the revision it was committed at. Then tag the suite and commit the tag,
+
+  .. code-block::
+
+    fcm pe fcm:revision .
+    <add tag to resulting editor>
+    fcm ci
+
+
+LFRic Apps Standard Suites
+--------------------------
+
+For LFRic Apps the following Suites must be updated:
+
+* LFRic Apps Azspice - `u-dn674 <https://code.metoffice.gov.uk/trac/roses-u/browser/d/n/6/7/4/trunk>`_
+* LFRic Apps EX1A - `u-dn704 <https://code.metoffice.gov.uk/trac/roses-u/browser/d/n/7/0/4/trunk>`_
+
+
+For each suite, do the following:
+
+* Get a copy of each of these suites and move into the working copy.
+
+    .. code-block::
+
+        rosie co u-dn704
+        cd ~/roses/u-dn704
+
+* In ``rose-suite.conf`` update the version number
+* Upgrade the metadata for the mesh and lfric_atm apps
+
+    .. code-block::
+
+
+        rose app-upgrade -a -y -C app/lfric_atm vnX.Y
+        rose app-upgrade -a -y -C app/mesh vnX.Y
+
+* Check the modules loaded in the ``flow.cylc`` for any changes vs. the Apps trunk.
+
+    * See ``rose-stem/site/meto/common/suite_config_PLATFORM.cylc``
+    * The lfric software stack moves more quickly than the UM, so it is more likely these have changed.
+
+* Test - ``cylc vip``
+* :ref:`Commit and Tag <suite_commit>` the suite once the test suite has succeeded.
+