Process aerosol analysis in grib2 format via UPP in GCAFS #4234
+257
−15
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../gfs/config.anal |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../gfs/config.analcalc |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| #! /usr/bin/env bash | ||
|
|
||
| ########## config.upp ########## | ||
| # UPP specific | ||
|
|
||
| echo "BEGIN: config.upp" | ||
|
|
||
| # Get task specific resources | ||
| . "${EXPDIR}/config.resources" upp | ||
|
|
||
| export UPP_CONFIG="${PARMgfs}/post/upp_gcafs.yaml" | ||
|
|
||
| # No. of forecast hours to process in a single job | ||
| export NFHRS_PER_GROUP=3 | ||
|
|
||
| echo "END: config.upp" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,28 @@ datasets: | |
| {% for itile in range(1,7) %} | ||
| - "{{ COMIN_CHEM_ANALYSIS | relpath(ROTDIR) }}/aeroinc.{{ cycle_YMD }}.{{ cycle_HH }}0000.fv_tracer.res.tile{{ itile }}.nc" | ||
| {% endfor %} | ||
| # Analysis Master GRIB2 data | ||
| - "{{ COMIN_ATMOS_MASTER | relpath(ROTDIR) }}/{{ head }}master.analysis.grib2" | ||
| - "{{ COMIN_ATMOS_MASTER | relpath(ROTDIR) }}/{{ head }}master.analysis.grib2.idx" | ||
|
|
||
| # Analysis GRIB2 (sub-sampled) data | ||
| - "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_a.0p25.analysis.grib2" | ||
| - "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_a.0p25.analysis.grib2.idx" | ||
| - "{{ COMIN_ATMOS_GRIB_0p50 | relpath(ROTDIR) }}/{{ head }}pres_a.0p50.analysis.grib2" | ||
| - "{{ COMIN_ATMOS_GRIB_0p50 | relpath(ROTDIR) }}/{{ head }}pres_a.0p50.analysis.grib2.idx" | ||
| - "{{ COMIN_ATMOS_GRIB_1p00 | relpath(ROTDIR) }}/{{ head }}pres_a.1p00.analysis.grib2" | ||
| - "{{ COMIN_ATMOS_GRIB_1p00 | relpath(ROTDIR) }}/{{ head }}pres_a.1p00.analysis.grib2.idx" | ||
| # temporarily disable pres_b products archiving. | ||
|
Contributor
There was a problem hiding this comment. @bbakernoaa note this for when we get your previous PR un-reverted |
||
| #- "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_b.0p25.analysis.grib2" | ||
| #- "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_b.0p25.analysis.grib2.idx" | ||
| #- "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_b.0p50.analysis.grib2" | ||
| #- "{{ COMIN_ATMOS_GRIB_0p25 | relpath(ROTDIR) }}/{{ head }}pres_b.0p50.analysis.grib2.idx" | ||
| #- "{{ COMIN_ATMOS_GRIB_1p00 | relpath(ROTDIR) }}/{{ head }}pres_b.1p00.analysis.grib2" | ||
| #- "{{ COMIN_ATMOS_GRIB_1p00 | relpath(ROTDIR) }}/{{ head }}pres_b.1p00.analysis.grib2.idx" | ||
|
|
||
| # Analysis netCDF (raw) data | ||
| - "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}analysis.atm.a006.nc" | ||
| - "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}analysis.sfc.a006.nc" | ||
| {% endif %} | ||
| {% if DO_PREP_OBS_AERO %} | ||
| - "{{ COMIN_OBS | relpath(ROTDIR) }}/{{ head }}aeroobs" | ||
|
|
@@ -65,6 +87,8 @@ datasets: | |
| - "{{ COMIN_ATMOS_RESTART | relpath(ROTDIR) }}/{{ r_prefix }}.coupler.res" | ||
| - "{{ COMIN_ATMOS_RESTART | relpath(ROTDIR) }}/{{ r_prefix }}.fv_core.res.nc" | ||
| {% endfor %} | ||
|
|
||
|
|
||
ypwang19 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| # Archive the EXPDIR if requested | ||
| {% if archive_expdir %} | ||
| {% filter indent(width=4) %} | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| upp: | ||
| config: | ||
| grib_version: "grib2" | ||
| ioform: "netcdfpara" | ||
| po: [1000.,975.,950.,925.,900.,875.,850.,825.,800.,775.,750.,725.,700.,675.,650.,625.,600.,575.,550.,525.,500.,475.,450.,425.,400.,375.,350.,325.,300.,275.,250.,225.,200.,175.,150.,125.,100.,70.,50.,40.,30.,20.,15.,10.,7.,5.,3.,2.,1.,0.7,0.4,0.2,0.1,0.07,0.04,0.02,0.01] | ||
| rdaod: False | ||
| nasa_on: False | ||
| fix_data: | ||
| mkdir: | ||
| - "{{ DATA }}" | ||
| copy: | ||
| - ["{{ 'g2tmpl_ROOT' | getenv }}/share/params_grib2_tbl_new", "{{ DATA }}/params_grib2_tbl_new"] | ||
| - ["{{ PARMgfs }}/post/nam_micro_lookup.dat", "{{ DATA }}/eta_micro_lookup.dat"] | ||
| - ["{{ EXECgfs }}/upp.x", "{{ DATA }}/"] | ||
| - ["{{ PARMgfs }}/post/itag.jinja", "{{ DATA }}/"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_DUST_nasa.dat", "{{ DATA }}/optics_luts_DUST_nasa.dat"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_NITR_nasa.dat", "{{ DATA }}/optics_luts_NITR_nasa.dat"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_SALT_nasa.dat", "{{ DATA }}/optics_luts_SALT_nasa.dat"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_SOOT_nasa.dat", "{{ DATA }}/optics_luts_SOOT_nasa.dat"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_SUSO_nasa.dat", "{{ DATA }}/optics_luts_SUSO_nasa.dat"] | ||
| - ["{{ PARMgfs }}/post/optics_luts_WASO_nasa.dat", "{{ DATA }}/optics_luts_WASO_nasa.dat"] | ||
bbakernoaa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| analysis: | ||
| config: | ||
| rdaod: False | ||
| nasa_on: True | ||
| NET: GFS # upp doesn't work with GCAFS,set to GFS instead | ||
| data_in: | ||
| copy: | ||
| - ["{{ PARMgfs }}/post/gcafs/postxconfig-NT-gcafs.txt", "{{ DATA }}/postxconfig-NT.txt"] | ||
| - ["{{ COMIN_ATMOS_ANALYSIS }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.analysis.atm.a006.nc", "{{ DATA }}/{{ atmos_filename }}"] | ||
| - ["{{ COMIN_ATMOS_ANALYSIS }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.analysis.sfc.a006.nc", "{{ DATA }}/{{ flux_filename }}"] | ||
| data_out: | ||
| copy: | ||
| - ["{{ DATA }}/GFSPRS.GrbF00", "{{ COMOUT_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.master.analysis.grib2"] | ||
| - ["{{ DATA }}/GFSPRS.GrbF00.idx", "{{ COMOUT_ATMOS_MASTER }}/{{ RUN }}.t{{ current_cycle | strftime('%H') }}z.master.analysis.grib2.idx"] | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| #!/usr/bin/env python | ||
CoryMartin-NOAA marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # calcanl_gcafs.py | ||
| # script to run executables to produce netCDF analysis | ||
| # on GCAFS gaussian grid for downstream users | ||
| # based on calcanl_gfs.py | ||
| import os | ||
| import datetime | ||
| from wxflow import FileHandler | ||
| from netCDF4 import Dataset, num2date | ||
| import numpy as np | ||
|
|
||
| python2fortran_bool = {True: '.true.', False: '.false.'} | ||
|
|
||
|
|
||
| # function to calculate analysis from two increment files and background | ||
| def calcanl_gcafs(RunDir, ComOut, APrefix): | ||
| print('calcanl_gcafs beginning at: ', datetime.datetime.utcnow()) | ||
| # add aerosol increments to background aerosol fields | ||
| # add meteorological increments to background meteorological fields | ||
TravisElless-NOAA marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # analysis increment is already assumed to be | ||
| # at the GCAFS C384 equivalent Gaussian resolution | ||
|
|
||
| # define path variables | ||
| inc_file = os.path.join(RunDir, 'siginc.nc') | ||
| anl_file = os.path.join(RunDir, 'siganl') | ||
| ges_file = os.path.join(RunDir, 'sigf06') | ||
|
|
||
| # add meteorological increments to background meteorological fields | ||
| metvars = [['spfh', 'sphum'], | ||
| ['tmp', 'T'], | ||
| ['ugrd', 'u'], | ||
| ['vgrd', 'v'], | ||
| ['dpres', 'delp'], | ||
| ['delz', 'delz'], | ||
| ['o3mr', 'o3mr'], | ||
| ['clwmr', 'liq_wat']] | ||
|
|
||
| with Dataset(inc_file, mode='r') as incfile, Dataset(ges_file, mode='r') as gesfile, Dataset(anl_file, mode='a') as anlfile: | ||
| # loop over meteorological variables and add increments to background | ||
| for ioname, incname in metvars: | ||
| print(f"Adding increment to background for variable: {ioname}") | ||
| bkg = gesfile.variables[ioname][:] | ||
| increment = incfile.variables[incname + '_inc'][:] | ||
| anl = bkg + np.flip(increment, axis=1) | ||
|
|
||
| anlfile.variables[ioname][:] = anl[:] | ||
|
|
||
| # handle pressfc as a special case | ||
| print("Adding increment to background for variable: pressfc") | ||
| # read bk attribute and compute ps_inc from delp_inc | ||
| bk = gesfile.ncattrs() | ||
| if 'bk' in bk: | ||
| bk = gesfile.getncattr('bk') | ||
| else: | ||
| # try to find bk as a variable if not an attribute | ||
| bk = gesfile.variables['bk'][:] | ||
|
|
||
| pressfc = gesfile.variables['pressfc'][:] | ||
| delp_inc = incfile.variables['delp_inc'][:] | ||
|
|
||
| # compute surface pressure increment | ||
| ps_inc = delp_inc[-1] / (bk[-1] - bk[-2]) | ||
|
|
||
| # add increment to background surface pressure | ||
| pressfc_anl = pressfc + np.flip(ps_inc, axis=0) | ||
| anlfile.variables['pressfc'][:] = pressfc_anl[:] | ||
|
|
||
| # add aerosol increments to background aerosol fields | ||
| aerovars = [['so4', 'mass_fraction_of_sulfate_in_air'], | ||
| ['bc1', 'mass_fraction_of_hydrophobic_black_carbon_in_air'], | ||
| ['bc2', 'mass_fraction_of_hydrophilic_black_carbon_in_air'], | ||
| ['oc1', 'mass_fraction_of_hydrophobic_organic_carbon_in_air'], | ||
| ['oc2', 'mass_fraction_of_hydrophilic_organic_carbon_in_air'], | ||
| ['dust1', 'mass_fraction_of_dust001_in_air'], | ||
| ['dust2', 'mass_fraction_of_dust002_in_air'], | ||
| ['dust3', 'mass_fraction_of_dust003_in_air'], | ||
| ['dust4', 'mass_fraction_of_dust004_in_air'], | ||
| ['dust5', 'mass_fraction_of_dust005_in_air'], | ||
| ['seas1', ''], # no seas1 increment | ||
| ['seas2', 'mass_fraction_of_sea_salt001_in_air'], | ||
| ['seas3', 'mass_fraction_of_sea_salt002_in_air'], | ||
| ['seas4', 'mass_fraction_of_sea_salt003_in_air'], | ||
| ['seas5', 'mass_fraction_of_sea_salt004_in_air'] | ||
| ] | ||
|
|
||
| inc_file = os.path.join(RunDir, 'aeroinc.nc') | ||
| with Dataset(inc_file, mode='r') as incfile, Dataset(ges_file, mode='r') as gesfile, Dataset(anl_file, mode='a') as anlfile: | ||
| for ioname, incname in aerovars: | ||
| print(f"Adding increment to background for variable: {ioname}") | ||
| bkg = gesfile.variables[ioname][:] | ||
| # no seas1 increment | ||
| if ioname == 'seas1': | ||
| anl = bkg | ||
| else: | ||
| increment = incfile.variables[incname][:] | ||
| # reordering the dimensions of increment (latitude, longitude, levels) to macth background (time, levs, lat, lon) | ||
| increment_reshape = np.transpose(increment, (2, 0, 1)) | ||
| anl = bkg + increment_reshape[np.newaxis, :, :, :] | ||
|
|
||
| anlfile.variables[ioname][:] = anl[:] | ||
| # update time (from 6 to 0) and time units in anlfile so UPP can create anl variables | ||
| time = gesfile.variables['time'] | ||
| time_val = time[:] | ||
| time_units = time.units | ||
| time_calendar = getattr(time, "calendar", "standard") | ||
| cycle_time = num2date(time_val, units=time_units, calendar=time_calendar) | ||
| time_units_new = f"hours since {cycle_time[0]}" | ||
| anlfile.variables['time'][:] = 0.0 | ||
| anlfile.variables['time'].setncattr("units", time_units_new) | ||
|
|
||
| print('calcanl_gcafs successfully completed at: ', datetime.datetime.utcnow()) | ||
|
|
||
|
|
||
| # run the function if this script is called from the command line | ||
| if __name__ == '__main__': | ||
| ComOut = os.getenv('COMOUT_ATMOS_ANALYSIS', './') | ||
| APrefix = os.getenv('APREFIX', '') | ||
| RunDir = os.getenv('DATA', './') | ||
|
|
||
| calcanl_gcafs(RunDir, ComOut, APrefix) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.