Add domain level to wave gridded COM path #3137

AntonMFernando-NOAA · 2024-12-03T16:40:08Z

Description

This pull request addresses the organization of wave gridded products within the file structure. Currently, all wave gridded products are placed in products/wave/gridded, without any further categorization. To align with the structure used by other component products (such as grib2), this update introduces a domain/resolution directory level to group output more effectively.
For example, instead of having all products in a single directory, the structure will now include specific directories based on domain and resolution (e.g., products/wave/gridded/arctic.9km, products/wave/gridded/global.0p16). This change improves organization and scalability for wave gridded product outputs.
Resolves Add domain level to wave gridded COM path #2677

Type of change

COM_WAVE_GRID_TMPL updated (or a new template, similar to COM_ATMOS_GRIB_TMP and COM_ATMOS_GRIB_GRID_TMPL) to include a directory level for the domain and resolution
Gridded wave output placed into separate directories based on domain/resolution within products/wave/gridded
Output otherwise remain unchanged

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO
Does this change require an update to any of the following submodules? NO

How has this been tested?

CI tests with HERA and HERCULES

Checklist

jobs/JGLOBAL_WAVE_POST_SBS

jobs/JGLOBAL_ARCHIVE

…NOAA/global-workflow into wavegrid_compath

jobs/JGLOBAL_ARCHIVE

jobs/JGLOBAL_WAVE_POST_SBS

AntonMFernando-NOAA · 2025-01-15T19:40:07Z

@JessicaMeixner-NOAA Ready to run for multi-grids.

DavidHuber-NOAA

Looks good!

WalterKolczynski-NOAA

wave_domain_grid.sh should only need to be sourced once in each job. For consistency, please place it at the job of the j-job right after the j-job header.

Other requests are minor, then we can get this thing tested.

WalterKolczynski-NOAA · 2025-01-16T06:18:33Z

jobs/JGLOBAL_WAVE_PRDGEN_GRIDDED

+    for grdID in ${GEMPAK_GRIDS}; do
+        source "${USHgfs}/wave_domain_grid.sh" 


Same as above.

Suggested change

for grdID in ${GEMPAK_GRIDS}; do

source "${USHgfs}/wave_domain_grid.sh"

for grdID in ${GEMPAK_GRIDS}; do

WalterKolczynski-NOAA · 2025-01-16T06:19:25Z

jobs/JGLOBAL_ARCHIVE

+        for grdID in ${wavepostGRD} ${waveinterpGRD}; do
+            source "${USHgfs}/wave_domain_grid.sh" 


Suggested change

for grdID in ${wavepostGRD} ${waveinterpGRD}; do

source "${USHgfs}/wave_domain_grid.sh"

for grdID in ${wavepostGRD} ${waveinterpGRD}; do

WalterKolczynski-NOAA · 2025-01-16T06:20:41Z

jobs/JGLOBAL_WAVE_POST_SBS

+    for grdID in ${wavepostGRD} ${waveinterpGRD}; do
+        source "${USHgfs}/wave_domain_grid.sh" 


Add the source "${USHgfs}/wave_domain_grid.sh" right after the jjob header.

Suggested change

for grdID in ${wavepostGRD} ${waveinterpGRD}; do

source "${USHgfs}/wave_domain_grid.sh"

for grdID in ${wavepostGRD} ${waveinterpGRD}; do

WalterKolczynski-NOAA · 2025-01-16T06:25:41Z

scripts/exgfs_wave_post_gridded_sbs.sh

+  gfile="${COMIN_WAVE_HISTORY}/${WAV_MOD_TAG}.out_grd.${waveGRD}.${YMD}.${HMS}"
+    if [[ ! -s "${gfile}" ]]; then
+      echo " FATAL ERROR : NO RAW FIELD OUTPUT FILE ${gfile}"
+      err=3; export err; "${errchk}"
+      exit "${err}"
    fi


Suggested change

gfile="${COMIN_WAVE_HISTORY}/${WAV_MOD_TAG}.out_grd.${waveGRD}.${YMD}.${HMS}"

if [[ ! -s "${gfile}" ]]; then

echo " FATAL ERROR : NO RAW FIELD OUTPUT FILE ${gfile}"

err=3; export err; "${errchk}"

exit "${err}"

fi

gfile="${COMIN_WAVE_HISTORY}/${WAV_MOD_TAG}.out_grd.${waveGRD}.${YMD}.${HMS}"

if [[ ! -s "${gfile}" ]]; then

echo " FATAL ERROR : NO RAW FIELD OUTPUT FILE ${gfile}"

err=3; export err; "${errchk}"

exit "${err}"

fi

jobs/JGLOBAL_ARCHIVE

WalterKolczynski-NOAA · 2025-01-16T06:32:17Z

parm/config/gfs/config.waveawipsgridded

@@ -8,6 +8,9 @@ echo "BEGIN: config.waveawipsgridded"
 # Get task specific resources
 . $EXPDIR/config.resources waveawipsgridded

+export DBNROOT=/dev/null


DBNROOT is set by config.base and definitely should not be changed.

Suggested change

export DBNROOT=/dev/null

WalterKolczynski-NOAA · 2025-01-16T06:33:06Z

scripts/exgfs_wave_post_gridded_sbs.sh

-        echo "${USHgfs}/wave_grib2_sbs.sh $grdID $GRIDNR $MODNR $ymdh $fhr $GRDNAME $GRDRES $gribFL > grib_$grdID.out 2>&1" >> ${fcmdnow}
-      done
-    fi
+        source "${USHgfs}/wave_domain_grid.sh"


Suggested change

source "${USHgfs}/wave_domain_grid.sh"

WalterKolczynski-NOAA · 2025-01-16T06:33:36Z

scripts/exgfs_wave_post_gridded_sbs.sh

+    for grdID in ${wavepostGRD} # First concatenate grib files for sbs grids
+    do
+      gribFL=\'$(echo ${OUTPARS_WAV})\'
+      source "${USHgfs}/wave_domain_grid.sh"


Suggested change

source "${USHgfs}/wave_domain_grid.sh"

WalterKolczynski-NOAA · 2025-01-16T06:33:50Z

scripts/exgfs_wave_prdgen_gridded.sh

-       echo "FATAL ERROR Unrecognized input grid ${grid_in}"
-       exit 2;;
- esac
+ source "${USHgfs}/wave_domain_grid.sh"


Suggested change

source "${USHgfs}/wave_domain_grid.sh"

WalterKolczynski-NOAA · 2025-01-16T06:34:07Z

scripts/exgfs_wave_prdgen_gridded.sh

-       *)       grdID= ;;
-     esac
-     #
+     source "${USHgfs}/wave_domain_grid.sh"


Suggested change

source "${USHgfs}/wave_domain_grid.sh"

…NOAA/global-workflow into wavegrid_compath

emcbot · 2025-01-16T20:53:11Z

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hera in Build# 1 with error logs:

/scratch1/NCEPDEV/global/CI/3137/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_8615cdff/logs/2020110100/gefs_extractvars_mem000.log
/scratch1/NCEPDEV/global/CI/3137/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_8615cdff/logs/2020110100/gefs_extractvars_mem001.log

Follow link here to view the contents of the above file(s): (link)

emcbot · 2025-01-16T20:53:21Z

Experiment C96_S2SWA_gefs_replay_ics FAILED on Hera in Build# 1 in
/scratch1/NCEPDEV/global/CI/3137/RUNTESTS/EXPDIR/C96_S2SWA_gefs_replay_ics_8615cdff

WalterKolczynski-NOAA · 2025-01-16T20:59:13Z

The extractvar job either needs to be removed or updated for the new gridded wave product path.

…NOAA/global-workflow into wavegrid_compath

emcbot · 2025-01-16T23:45:42Z

CI Failed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/CI/3137


Experiment C96_S2SWA_gefs_replay_ics_8615cdff Terminated with 0
FAIL
FAIL tasks failed and 2 dead at Thu Jan 16 20:52:47 UTC 2025
Experiment C96_S2SWA_gefs_replay_ics_8615cdff Terminated: *FAIL*
Error logs:
/scratch1/NCEPDEV/global/CI/3137/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_8615cdff/logs/2020110100/gefs_extractvars_mem000.log
/scratch1/NCEPDEV/global/CI/3137/RUNTESTS/COMROOT/C96_S2SWA_gefs_replay_ics_8615cdff/logs/2020110100/gefs_extractvars_mem001.log
Experiment C48mx500_3DVarAOWCDA_8615cdff Completed 2 Cycles: *SUCCESS* at Thu Jan 16 21:35:35 UTC 2025
Experiment C48_ATM_8615cdff Completed 2 Cycles: *SUCCESS* at Thu Jan 16 21:53:56 UTC 2025
Experiment C48mx500_hybAOWCDA_8615cdff Completed 2 Cycles: *SUCCESS* at Thu Jan 16 21:53:56 UTC 2025
Experiment C96C48_hybatmaerosnowDA_8615cdff Completed 3 Cycles: *SUCCESS* at Thu Jan 16 23:02:06 UTC 2025
Experiment C48_S2SWA_gefs_8615cdff Completed 1 Cycles: *SUCCESS* at Thu Jan 16 23:06:26 UTC 2025
Experiment C96_atm3DVar_8615cdff Completed 3 Cycles: *SUCCESS* at Thu Jan 16 23:08:32 UTC 2025
Experiment C96C48_hybatmDA_8615cdff Completed 3 Cycles: *SUCCESS* at Thu Jan 16 23:14:48 UTC 2025
Experiment C48_S2SW_8615cdff Completed 2 Cycles: *SUCCESS* at Thu Jan 16 23:20:54 UTC 2025
Experiment C96C48_ufs_hybatmDA_8615cdff Completed 3 Cycles: *SUCCESS* at Thu Jan 16 23:45:29 UTC 2025

AntonMFernando-NOAA added 4 commits December 3, 2024 11:29

modified config.com

b6a732e

modified jobs\JGLOBAL_ARCHIVE

0a5ca0e

modified jobs\JGLOBAL_EXTRACTVARS

cb496d1

modified jobs\JGLOBAL_WAVE_POST_SBS

3492b8c

JessicaMeixner-NOAA reviewed Dec 3, 2024

View reviewed changes

jobs/JGLOBAL_WAVE_POST_SBS Outdated Show resolved Hide resolved

AntonMFernando-NOAA and others added 3 commits December 4, 2024 12:19

modified jobs\JGLOBAL_WAVE_PRDGEN_GRIDDED

92cba9c

updated the relevant yaml files

a0aa55b

Merge branch 'NOAA-EMC:develop' into wavegrid_compath

fe0f375

WalterKolczynski-NOAA reviewed Dec 4, 2024

View reviewed changes

jobs/JGLOBAL_ARCHIVE Outdated Show resolved Hide resolved

AntonMFernando-NOAA and others added 4 commits December 4, 2024 14:43

modified parm\archive\gefs_extracted_wave.yaml.j2

52eda4b

Merge branch 'NOAA-EMC:develop' into wavegrid_compath

2f220ea

modified jobs\JGLOBAL_ARCHIVE

c2ce78f

Merge branch 'wavegrid_compath' of https://github.com/AntonMFernando-…

403a395

…NOAA/global-workflow into wavegrid_compath

github-advanced-security bot found potential problems Dec 5, 2024

View reviewed changes

jobs/JGLOBAL_ARCHIVE Fixed Show fixed Hide fixed

jobs/JGLOBAL_ARCHIVE Fixed Show fixed Hide fixed

modified jobs\JGLOBAL_ARCHIVE

261c833

github-advanced-security bot found potential problems Dec 5, 2024

View reviewed changes

jobs/JGLOBAL_ARCHIVE Fixed Show fixed Hide fixed

AntonMFernando-NOAA added 6 commits December 5, 2024 11:48

modified jobs\JGLOBAL_ARCHIVE

8b01949

modified typos JGLOBAL_ARCHIVE and JGLOBAL_WAVE_POST_SBS

257cd4a

modifed jobs\JGLOBAL_ARCHIVE

4674706

modified jobs\JGLOBAL_ARCHIVE

bac1bb9

modified jobs\JGLOBAL_EXTRACTVARS

477df47

modified jobs\JGLOBAL_WAVE_POST_SBS

f1e54ba

github-advanced-security bot found potential problems Dec 5, 2024

View reviewed changes

jobs/JGLOBAL_WAVE_POST_SBS Fixed Show fixed Hide fixed

AntonMFernando-NOAA added 7 commits December 5, 2024 12:31

modified jobs\JGLOBAL_WAVE_POST_SBS

a91797e

modified jobs\JGLOBAL_WAVE_POST_SBS

832141f

modified JGLOBAL

cccf13a

modified JGLOBAL jobs

2ce47a6

modified jobs\JGLOBAL_ARCHIVE

fd3e0e4

modified jobs\JGLOBAL_WAVE_PRDGEN_GRIDDED

75bb84a

modified JGLOBAl scripts

acee85c

DavidHuber-NOAA previously approved these changes Jan 15, 2025

View reviewed changes

WalterKolczynski-NOAA requested changes Jan 16, 2025

View reviewed changes

AntonMFernando-NOAA and others added 3 commits January 16, 2025 11:04

Merge branch 'NOAA-EMC:develop' into wavegrid_compath

2f06df0

modified parm\config\gfs\config.waveawipsgridded

2b511c2

Merge branch 'wavegrid_compath' of https://github.com/AntonMFernando-…

e37bc87

…NOAA/global-workflow into wavegrid_compath

AntonMFernando-NOAA dismissed DavidHuber-NOAA’s stale review via e37bc87 January 16, 2025 17:10

AntonMFernando-NOAA added 2 commits January 16, 2025 12:18

moved source wave_domain_grid.sh

083c26b

typo fixed scripts\exgfs_wave_post_gridded_sbs.sh

8615cdf

AntonMFernando-NOAA requested a review from WalterKolczynski-NOAA January 16, 2025 17:25

WalterKolczynski-NOAA previously approved these changes Jan 16, 2025

View reviewed changes

WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Jan 16, 2025

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Jan 16, 2025

AntonMFernando-NOAA and others added 2 commits January 16, 2025 16:12

Merge branch 'develop' into wavegrid_compath

8e34b37

modified extractvars files

1adfe47

AntonMFernando-NOAA dismissed WalterKolczynski-NOAA’s stale review via 1adfe47 January 16, 2025 22:12

AntonMFernando-NOAA and others added 3 commits January 16, 2025 17:13

Merge branch 'develop' into wavegrid_compath

d406b17

modified GEMPAK files

370840f

Merge branch 'wavegrid_compath' of https://github.com/AntonMFernando-…

508dfb4

…NOAA/global-workflow into wavegrid_compath

emcbot added CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed and removed CI-Hera-Failed **Bot use only** CI testing on Hera for this PR has failed labels Jan 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add domain level to wave gridded COM path #3137

Add domain level to wave gridded COM path #3137

AntonMFernando-NOAA commented Dec 3, 2024 •

edited

Loading

AntonMFernando-NOAA commented Jan 15, 2025

DavidHuber-NOAA left a comment

WalterKolczynski-NOAA left a comment

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

WalterKolczynski-NOAA Jan 16, 2025

emcbot commented Jan 16, 2025

emcbot commented Jan 16, 2025

WalterKolczynski-NOAA commented Jan 16, 2025

emcbot commented Jan 16, 2025

		for grdID in ${GEMPAK_GRIDS}; do
		source "${USHgfs}/wave_domain_grid.sh"

		for grdID in ${wavepostGRD} ${waveinterpGRD}; do
		source "${USHgfs}/wave_domain_grid.sh"

Add domain level to wave gridded COM path #3137

Are you sure you want to change the base?

Add domain level to wave gridded COM path #3137

Conversation

AntonMFernando-NOAA commented Dec 3, 2024 • edited Loading

Description

Type of change

Change characteristics

How has this been tested?

Checklist

AntonMFernando-NOAA commented Jan 15, 2025

DavidHuber-NOAA left a comment

Choose a reason for hiding this comment

WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emcbot commented Jan 16, 2025

emcbot commented Jan 16, 2025

WalterKolczynski-NOAA commented Jan 16, 2025

emcbot commented Jan 16, 2025

AntonMFernando-NOAA commented Dec 3, 2024 •

edited

Loading