Skip to content

Commit

Permalink
Enable some of the intel llvm tests in rt conf + Second reconciliatio…
Browse files Browse the repository at this point in the history 
…n PR from production/RRFS.v1 ufs-community#2481 (ufs-community#2458)

* UFSWM - Enable some of the intel llvm tests in rt conf
* UFSWM - Second reconciliation PR from production/RRFS.v1 related to FV3 unused and optional variables 
  * FV3 - Second reconciliation PR from production/RRFS.v1 related to FV3 unused and optional variables
    * ccpp-physics - Second reconciliation PR from production/RRFS.v1 related to FV3 unused and optional variables

---------

Co-authored-by: Grant Firl <[email protected]>
  • Loading branch information
@DusanJovic-NOAA @grantfirl
DusanJovic-NOAA and grantfirl authored Nov 8, 2024
1 parent f9c91d3 commit 058f073
Show file tree
Hide file tree
Showing 14 changed files with 1,694 additions and 1,831 deletions.
2 changes: 1 addition & 1 deletion FV3
Submodule FV3 updated 6 files
+0 −47 ccpp/data/CCPP_typedefs.F90
+10 −19 ccpp/data/CCPP_typedefs.meta
+32 −85 ccpp/data/GFS_typedefs.F90
+85 −88 ccpp/data/GFS_typedefs.meta
+1 −87 ccpp/driver/GFS_diagnostics.F90
+1 −1 ccpp/physics
2 changes: 1 addition & 1 deletion modulefiles/ufs_gaea.intel.lua
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
help([[
This module loads libraries required for building and running UFS Weather Model
This module loads libraries required for building and running UFS Weather Model
on the NOAA RDHPC machine Gaea C5 using Intel-2023.1.0.
]])

Expand Down
11 changes: 5 additions & 6 deletions modulefiles/ufs_gaea.intelllvm.lua
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
help([[
This module loads libraries required for building and running UFS Weather Model
This module loads libraries required for building and running UFS Weather Model
on the NOAA RDHPC machine Gaea C5 using Intel-2023.1.0.
]])

whatis([===[Loads libraries needed for building the UFS Weather Model on Gaea ]===])

prepend_path("MODULEPATH", "/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core")

stack_intel_ver=os.getenv("stack_intel_ver") or "2023.1.0"
stack_intel_ver=os.getenv("stack_intel_ver") or "2023.2.0"
load(pathJoin("stack-intel", stack_intel_ver))

stack_cray_mpich_ver=os.getenv("stack_cray_mpich_ver") or "8.1.25"
stack_cray_mpich_ver=os.getenv("stack_cray_mpich_ver") or "8.1.28"
load(pathJoin("stack-cray-mpich", stack_cray_mpich_ver))

stack_python_ver=os.getenv("stack_python_ver") or "3.10.13"
Expand All @@ -27,8 +27,7 @@ load(pathJoin("nccmp", nccmp_ver))
unload("darshan-runtime")
unload("cray-libsci")

unload("intel-classic/2023.1.0")
load("intel-oneapi/2023.1.0")
load("intel-classic/2023.2.0")

setenv("I_MPI_CC", "icx")
setenv("I_MPI_CXX", "icpx")
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/ufs_hera.intelllvm.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ help([[
loads UFS Model prerequisites for Hera/IntelLLVM
]])

prepend_path("MODULEPATH", "/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core")

stack_intel_ver=os.getenv("stack_intel_ver") or "2021.5.0"
load(pathJoin("stack-intel", stack_intel_ver))
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/ufs_hercules.intelllvm.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ help([[
loads UFS Model prerequisites for Hercules/IntelLLVM
]])

prepend_path("MODULEPATH", "/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core")

stack_intel_ver=os.getenv("stack_intel_ver") or "2021.9.0"
load(pathJoin("stack-intel", stack_intel_ver))
Expand Down
2 changes: 1 addition & 1 deletion modulefiles/ufs_orion.intelllvm.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ help([[
loads UFS Model prerequisites for OrionLLVM/Intel
]])

prepend_path("MODULEPATH", "/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core")
prepend_path("MODULEPATH", "/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core")

stack_intel_ver=os.getenv("stack_intel_ver") or "2021.9.0"
load(pathJoin("stack-intel", stack_intel_ver))
Expand Down
578 changes: 295 additions & 283 deletions tests/logs/RegressionTests_gaea.log
View file

Large diffs are not rendered by default.

748 changes: 377 additions & 371 deletions tests/logs/RegressionTests_hera.log
View file

Large diffs are not rendered by default.

746 changes: 376 additions & 370 deletions tests/logs/RegressionTests_hercules.log
View file

Large diffs are not rendered by default.

566 changes: 249 additions & 317 deletions tests/logs/RegressionTests_jet.log
View file

Large diffs are not rendered by default.

572 changes: 289 additions & 283 deletions tests/logs/RegressionTests_orion.log
View file

Large diffs are not rendered by default.

99 changes: 73 additions & 26 deletions tests/rt.conf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@
# Item 4: Controls whether the run creates its own baseline or it uses the baseline from a different (control) test.
# Item 5: Test name to compare baselines with if not itself.

### Intel Tests ###
#############################
### Intel (Classic) Tests ###
#############################

### S2S tests ###
COMPILE | s2swa_32bit | intel | -DAPP=S2SWA -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8,FV3_GFS_v17_coupled_p8_ugwpv1 | | fv3 |
RUN | cpld_control_p8_mixedmode | - noaacloud | baseline |
Expand Down Expand Up @@ -146,12 +149,9 @@ COMPILE | csawmg | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16_csawmg,FV3_GFS_v1
RUN | control_csawmg | - noaacloud | baseline |
RUN | control_ras | - noaacloud | baseline |

COMPILE | csawmg | gnu | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16_csawmg,FV3_GFS_v16_ras | + hera hercules | fv3 |
RUN | control_csawmg | + hera hercules | baseline |

# Run WAM test in REPRO mode to avoid numerical instability in the deep atmosphere
COMPILE | wam | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8 -D32BIT=ON -DMULTI_GASES=ON | - noaacloud | fv3 |
RUN | control_wam | - noaacloud | baseline |
RUN | control_wam | - noaacloud | baseline |

COMPILE | atm_faster_dyn32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8,FV3_GFS_v15_thompson_mynn_lam3km -D32BIT=ON -DFASTER=ON | | fv3 |
RUN | control_p8_faster | - noaacloud | baseline |
Expand Down Expand Up @@ -190,10 +190,10 @@ COMPILE | atm_debug_dyn32 | gnu | -DAPP=ATM -DDEBUG=ON -D32BIT=ON -DCCPP_SUITES=
RUN | control_csawmg_debug | + hera hercules | baseline |

COMPILE | wam_debug | intel | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8 -D32BIT=ON -DMULTI_GASES=ON -DDEBUG=ON | - noaacloud | fv3 |
RUN | control_wam_debug | - noaacloud hercules | baseline |
RUN | control_wam_debug | - noaacloud hercules | baseline |

### 32-bit physics tests ###
COMPILE | rrfs_dyn32_phy32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR -D32BIT=ON -DCCPP_32BIT=ON | - noaacloud | fv3 |
COMPILE | rrfs_dyn32_phy32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR -D32BIT=ON -DCCPP_32BIT=ON | - noaacloud | fv3 |
RUN | regional_spp_sppt_shum_skeb_dyn32_phy32 | - noaacloud | baseline |
RUN | rap_control_dyn32_phy32 | - noaacloud | baseline |
RUN | hrrr_control_dyn32_phy32 | - noaacloud | baseline |
Expand All @@ -213,21 +213,21 @@ RUN | conus13km_restart_mismatch | - noaacloud
# RUN | conus13km_decomp | - noaacloud | | conus13km_control

COMPILE | rrfs_dyn64_phy32 | intel | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR -DCCPP_32BIT=ON | - noaacloud | fv3 |
RUN | rap_control_dyn64_phy32 | - noaacloud | baseline |
RUN | rap_control_dyn64_phy32 | - noaacloud | baseline |

COMPILE | rrfs_dyn32_phy32_debug | intel | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR,FV3_HRRR_gf -D32BIT=ON -DCCPP_32BIT=ON -DDEBUG=ON | - noaacloud | fv3 |
RUN | rap_control_debug_dyn32_phy32 | - noaacloud | baseline |
RUN | hrrr_control_debug_dyn32_phy32 | - noaacloud | baseline |
RUN | conus13km_debug | - noaacloud | baseline |
RUN | conus13km_debug_qr | - noaacloud | |
RUN | conus13km_debug_2threads | - noaacloud | |
RUN | conus13km_radar_tten_debug | - noaacloud | baseline |
RUN | rap_control_debug_dyn32_phy32 | - noaacloud | baseline |
RUN | hrrr_control_debug_dyn32_phy32 | - noaacloud | baseline |
RUN | conus13km_debug | - noaacloud | baseline |
RUN | conus13km_debug_qr | - noaacloud | |
RUN | conus13km_debug_2threads | - noaacloud | |
RUN | conus13km_radar_tten_debug | - noaacloud | baseline |

# Expected to fail:
# RUN | conus13km_debug_decomp | - noaacloud | |
# RUN | conus13km_debug_decomp | - noaacloud | |

COMPILE | rrfs_dyn64_phy32_debug | intel | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR -DCCPP_32BIT=ON -DDEBUG=ON | - noaacloud | fv3 |
RUN | rap_control_dyn64_phy32_debug | - noaacloud | baseline |
RUN | rap_control_dyn64_phy32_debug | - noaacloud | baseline |

### HAFS tests ###
COMPILE | hafsw | intel | -DAPP=HAFSW -DMOVING_NEST=ON -DCCPP_SUITES=FV3_HAFS_v1_gfdlmp_tedmf,FV3_HAFS_v1_gfdlmp_tedmf_nonsst,FV3_HAFS_v1_thompson_tedmf_gfdlsf,FV3_global_nest_v1 -D32BIT=ON | | fv3 |
Expand Down Expand Up @@ -327,9 +327,59 @@ RUN | atmaero_control_p8_rad_micro | - noaacloud
#RUN | regional_atmaq | - jet s4 | baseline |

COMPILE | atmaq_debug | intel | -DAPP=ATMAQ -DCCPP_SUITES=FV3_GFS_v15p2 -DDEBUG=ON -D32BIT=ON | - jet noaacloud s4 | fv3 |
RUN | regional_atmaq_debug | - hera jet s4 noaacloud | baseline |
RUN | regional_atmaq_debug | - hera jet s4 noaacloud | baseline |

### ATM-FBH test ###
COMPILE | atm_fbh | intel | -DAPP=ATMF -DCCPP_SUITES=FV3_HRRR -D32BIT=ON | - wcoss2 noaacloud acorn | fv3 |
RUN | cpld_regional_atm_fbh | - wcoss2 noaacloud acorn | baseline |



#############################
### Intel (LLVM) Tests ######
#############################


# Coupled gfs17 tests on Gaea
COMPILE | s2swa_32bit_pdlib | intelllvm | -DAPP=S2SWA -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON | + gaea | fv3 |
RUN | cpld_control_gfsv17 | + gaea | baseline |

COMPILE | s2swa_32bit_pdlib_debug | intelllvm | -DAPP=S2SWA -D32BIT=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON -DDEBUG=ON | + gaea | fv3 |
RUN | cpld_debug_gfsv17 | + gaea | baseline |

COMPILE | s2swa_32bit_pdlib_sfs | intelllvm | -DAPP=S2SWA -D32BIT=ON -DHYDRO=ON -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON | + gaea | fv3 |
RUN | cpld_control_sfs | + gaea | baseline |

COMPILE | s2swa | intelllvm | -DAPP=S2SWA -DCCPP_SUITES=FV3_GFS_v17_coupled_p8,FV3_GFS_v17_coupled_p8_ugwpv1 | + gaea | fv3 |
RUN | cpld_control_p8 | + gaea | baseline |

# Regional RRFS tests on Hera
COMPILE | rrfs_dyn32_phy32 | intelllvm | -DAPP=ATM -DCCPP_SUITES=FV3_RAP -D32BIT=ON -DCCPP_32BIT=ON | + hera | fv3 |
RUN | rap_control_dyn32_phy32 | + hera | baseline |

COMPILE | rrfs_dyn64_phy32 | intelllvm | -DAPP=ATM -DCCPP_SUITES=FV3_RAP -DCCPP_32BIT=ON | + hera | fv3 |
RUN | rap_control_dyn64_phy32 | + hera | baseline |

# HAFS tests on Orion
COMPILE | hafsw | intelllvm | -DAPP=HAFSW -DMOVING_NEST=ON -DCCPP_SUITES=FV3_HAFS_v1_gfdlmp_tedmf_nonsst -D32BIT=ON | + orion | fv3 |
RUN | hafs_regional_storm_following_1nest_atm_ocn | + orion | baseline |

COMPILE | hafsw_debug | intelllvm | -DAPP=HAFSW -DMOVING_NEST=ON -DCCPP_SUITES=FV3_HAFS_v1_gfdlmp_tedmf_nonsst -D32BIT=ON -DDEBUG=ON | + orion | fv3 |
RUN | hafs_regional_storm_following_1nest_atm_ocn_debug | + orion | baseline |

# DATM/CDEPS tests on Hercules
COMPILE | datm_cdeps | intelllvm | -DAPP=NG-GODAS | + hercules | fv3 |
RUN | datm_cdeps_control_cfsr | + hercules | baseline |

COMPILE | datm_cdeps_debug | intelllvm | -DAPP=NG-GODAS -DDEBUG=ON | + hercules | fv3 |
RUN | datm_cdeps_debug_cfsr | + hercules | baseline |



#############################
### Gnu Tests ###############
#############################

### GNU TESTS ###
### CCPP PROD tests ###
COMPILE | atm | gnu | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16,FV3_GFS_v16_flake,FV3_GFS_v16_ras,FV3_GFS_v17_p8,FV3_GFS_v17_p8_ugwpv1 | + hera hercules | fv3 |
RUN | control_c48 | + hera hercules | baseline |
Expand All @@ -355,6 +405,9 @@ RUN | hrrr_control_restart | + hera hercules
RUN | hrrr_control_restart_noqr | + hera hercules | | hrrr_control_noqr
RUN | rrfs_v1beta | + hera hercules | baseline |

COMPILE | csawmg | gnu | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v16_csawmg,FV3_GFS_v16_ras | + hera hercules | fv3 |
RUN | control_csawmg | + hera hercules | baseline |

### CCPP DEBUG tests ###
# Exercise compilation without specifying suites (i.e. compile all suites) in DEBUG mode (faster than in PROD mode)
COMPILE | atm_dyn32_debug | gnu | -DAPP=ATM -D32BIT=ON -DDEBUG=ON | + hera hercules | fv3 |
Expand All @@ -375,10 +428,8 @@ RUN | rap_flake_debug | + hera hercules
RUN | rap_clm_lake_debug | + hera hercules | baseline |
RUN | gnv1_c96_no_nest_debug | + hera hercules | baseline |


COMPILE | wam_debug | gnu | -DAPP=ATM -DCCPP_SUITES=FV3_GFS_v17_p8 -D32BIT=ON -DMULTI_GASES=ON -DDEBUG=ON | + hera hercules | fv3 |
RUN | control_wam_debug | + hercules | baseline |

RUN | control_wam_debug | + hercules | baseline |

### 32-bit physics tests ###
COMPILE | rrfs_dyn32_phy32 | gnu | -DAPP=ATM -DCCPP_SUITES=FV3_RAP,FV3_HRRR -D32BIT=ON -DCCPP_32BIT=ON | + hera hercules | fv3 |
Expand Down Expand Up @@ -429,12 +480,8 @@ COMPILE | s2sw_pdlib | gnu | -DAPP=S2SW -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugw
RUN | cpld_control_pdlib_p8 | + hera hercules | baseline |

COMPILE | s2sw_pdlib_debug | gnu | -DAPP=S2SW -DCCPP_SUITES=FV3_GFS_v17_coupled_p8_ugwpv1 -DPDLIB=ON -DDEBUG=ON | + hera hercules | fv3 |
RUN | cpld_debug_pdlib_p8 | + hercules | baseline |
RUN | cpld_debug_pdlib_p8 | + hercules | baseline |

### CDEPS Data Atmosphere test ###
COMPILE | datm_cdeps | gnu | -DAPP=NG-GODAS | + hera hercules | fv3 |
RUN | datm_cdeps_control_cfsr | + hera hercules | baseline |

### ATM-FBH test ###
COMPILE | atm_fbh | intel | -DAPP=ATMF -DCCPP_SUITES=FV3_HRRR -D32BIT=ON | - wcoss2 noaacloud acorn | fv3 |
RUN | cpld_regional_atm_fbh | - wcoss2 noaacloud acorn | baseline |
Loading

0 comments on commit 058f073

Please sign in to comment.