Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add JASMIN as an additional site to the RTW #3302

Merged
merged 45 commits into from
Oct 20, 2023
Merged

Add JASMIN as an additional site to the RTW #3302

merged 45 commits into from
Oct 20, 2023

Conversation

ehogan
Copy link
Contributor

@ehogan ehogan commented Jul 19, 2023 •
edited
Loading

Description

Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

To help with the number of pull requests:

@mo-tgeddes
#3060: add jasmin framework
69ae508
@mo-tgeddes
#3060: add jasmin specific requirements
b06abbf
@mo-tgeddes
#3060: modify platform settings
c916ea4
@mo-tgeddes
#3060: modify platform settings
ca07fd5
@mo-tgeddes
#3060: modify directives
4509345
@mo-tgeddes
#3060: add cloning urls to site specific config files
bf6733c
@mo-tgeddes
#3060: add cloning urls to general config
8cb0cf2
@mo-tgeddes
#3060: add cloning urls to flow.cylc
99708d3
@mo-tgeddes
#3060: comment out install image hash
92bf187
@mo-tgeddes
#3060: make rtw file executable
b0cea51
@mo-tgeddes
#3060: remove jasmin-env
80f7d48
@mo-tgeddes
#3060: add jasmin-env again
3d735c8
@mo-tgeddes
#3060 remove jasmin.cylc
e13b95d
@mo-tgeddes
#3060 add jasmin.cylc
fe01656
@mo-tgeddes
#3060: change localhost to batch system
1f6d198
@mo-tgeddes
#3060: change batch to platform
522a480
@mo-tgeddes
#3060: change batch to platform
ac082cd
@mo-tgeddes
#3060: update max tasks and time limit
b1242bb
@mo-tgeddes
#3060: add batch system back
6fb55c6
@mo-tgeddes
#3060: add method = slurm
03c7df7
@mo-tgeddes
#3060: add job submission
85eb16d
@mo-tgeddes
#3060: put it back to how it was
476040b
@mo-tgeddes
#3060: remove platform
4115897
@mo-tgeddes
#3060: revert changes back to working copy
f042102
@mo-tgeddes
#3060: revert changes back to working copy
5645c56
@mo-tgeddes
#3060: remove host specification
c35b24c
@mo-tgeddes
#3060: add localhost declaration
6462efc
@mo-tgeddes
#3060: chang max tasks
184625a
@mo-tgeddes
#3060: get rid of the jasmin specific things
3331e1e
@mo-tgeddes
#3060: checkout to non broken commit
0ce8576
@ehogan ehogan self-assigned this Jul 19, 2023
@ehogan
Merge branch 'recipe_test_workflow_prototype' into 3060_add_jasmin_to…
95ae33b 
…_RTW

* recipe_test_workflow_prototype: (237 commits)
  Add version to dataset in python example recipe to avoid "Unknown file format" issue on JASMIN (#3322)
  CMORizer for NASA MERRA reanalysis (#3039)
  Add `OBS-maintainers` team to documentation on OBS data maintenance and CMORizer reviews (#3335)
  Fixed provenance tracking for NCL multipanel PNGs (#3332)
  Cmorizer for NOAA-CIRES-20CR v3 reanalysis (clt, clwvi, hus, prw, rlut, rlutcs, rsut, rsutcs) (#3137)
  [Condalock] Update Linux condalock file (#3321)
  Slight refactoring of diag `galytska23/select_variables_for_tigramite.py` for generality and portability (for Changelog v2.10: authors: @valeriupredoi and @egalytska) (#3298)
  Removed recipe_carvalhais14nat from list of broken recipes (#3319)
  add Romain Beucher to CITATION as contributor (#3318)
  update `mamba` version in readthedocs configuration docs builds (#3310)
  [Github Actions] Compress all bash shell setters into one default option per workflow (#3315)
  [condalock] update conda lock creation Github Action workflow and ship updated (bot-generated) conda-lock file (#3307)
  Allow NCL unit conversion `kg s-1` -> `GtC y-1` (#3300)
  Add list of failing recipes for v2.9.0 release (#3294)
  Update diag_shapeselect.py to work with shapely v2 (#3283)
  Update release schedule after release of v2.9.0 (#3289)
  Add merge instructions to release instructions (#3292)
  Made sklearn test backwards-compatible with sklearn < 1.3 (#3285)
  Add release notes for v2.9 (#3266)
  Add release notes for v2.9 (#3266)
  ...
@ehogan ehogan added the Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow label Aug 30, 2023
@ehogan ehogan linked an issue Aug 30, 2023 that may be closed by this pull request
Add JASMIN as an additional site to the RTW #3060
Closed
@ehogan ehogan changed the title Add JASMIN as an additional site Add JASMIN as an additional site to the RTW Oct 6, 2023
@ehogan ehogan marked this pull request as ready for review October 17, 2023 16:19
alistairsellar
alistairsellar approved these changes Oct 18, 2023
View reviewed changes
Copy link
Contributor

@alistairsellar alistairsellar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff. Branch works for me on Jasmin.

esmvaltool/utils/recipe_test_workflow/recipe_test_workflow/opt/rose-suite-metoffice.conf Outdated
@@ -1,7 +1,7 @@
[template variables]
DRS_CMIP5="BADC"
DRS_CMIP6="BADC"
KGO_ROOT_PATH="/data/users/esmval/KGO/"
KGO_ROOT_PATH="/data/users/esmval/KGO_v2.6.0/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this comment out of scope, but consider upgrading this to KGO_v2.9.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done in #3386 and has just been merged into this branch :)

@ehogan
Copy link
Contributor Author

ehogan commented Oct 18, 2023

Hi @alistairsellar, I just remembered I wanted to add how to run the RTW on JASMIN to the documentation. I have now done that via 49ac5ad :) Does this look ok to you / did you do anything different when running the RTW on JASMIN?

@alistairsellar
Copy link
Contributor

Hi @alistairsellar, I just remembered I wanted to add how to run the RTW on JASMIN to the documentation. I have now done that via 49ac5ad :) Does this look ok to you / did you do anything different when running the RTW on JASMIN?

These instructions are what I followed, though I did need to

cd ESMValTool/esmvaltool/utils/recipe_test_workflow/recipe_test_workflow

after ssh to cylc, since you start at $HOME. So perhaps add this cd to the instructions, or a note that it needs to be run in the cloned RTW directory.

@ehogan
Copy link
Contributor Author

ehogan commented Oct 20, 2023

Thanks @alistairsellar! I hadn't included it as I thought the mention of it in a previous section was sufficient, but I forgot the path resets after ssh-ing! Now added: 63e8a73 :)

@ehogan ehogan merged commit 985732c into recipe_test_workflow_prototype Oct 20, 2023
8 checks passed
@ehogan ehogan deleted the 3060_add_jasmin_to_RTW branch October 20, 2023 15:15
@ehogan ehogan mentioned this pull request Apr 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alistairsellar alistairsellar alistairsellar approved these changes

Assignees

@ehogan ehogan

Labels
Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add JASMIN as an additional site to the RTW
3 participants
@ehogan @alistairsellar @mo-tgeddes