GGG2020.1 release checklist

[joshua-laughner]

This is a tracking issue to ensure that all code updates for GGG2020.1 are incorporated.

• Ensure the observation operator includes the new O2 DMF
• Ensure old added quantities (x2019 CO2, old obs operator) are not in the locally-produced .private.nc files.
• Ensure I2S changes for nonlinearity coefficients merged
• Ensure AK calculation from Jacobian files merged
• Ensure netCDF writer includes GEOS source information in uploaded files
• Update file format version in netCDF writer
• Ensure that the benchmark .mod and .vmr files include the GEOS source information so that the benchmark case tests that that information makes it through
• Update output formats to avoid quantization in some values
• Ensure that Dave G's bugfix for I2S firmware is merged
• Update I2S to handle alternate EM27 instrument header values (GGGBugs issue 345)
• Ensure issue Path length + periods issue #1 is addressed
• Remove -l from check_python.sh (GGGBugs issue 354)
• Rename the various "master" scripts
• Remove the "defaults" channel from the Python environment
• Fork netCDF writer to TCCON GitHub and repoint the install clone script there
• Update the benchmark files.

[joshua-laughner]

The path length + periods fix (#1) is implemented in Mercurial changeset f67568a14514

[joshua-laughner]

Alternate EM27 instrument strings (GGGBugs issue 345) addressed in Mercurial changeset 104138765dc7.

[joshua-laughner]

The firmware bugfix from GGGBugs issue 349 is implemented in Mercurial changeset 2900d96ad202.

[joshua-laughner]

Confirmed that a fresh clone of the GGG development repository can create AKs using the new method (Jacobian files, then AKs). Updated runme_install_main.sh in Mercurial changeset c419f19efca4.

[joshua-laughner]

I2S nonlinearity coefficients:

• Confirmed that the spectra created during the benchmark testing include the NCF and NC1 to NC4 header values
• Confirmed that the example opus-i2s and slice-i2s input files include the new parameter.

[joshua-laughner]

Increased precision as of Mercurial changesets 7953da8e71cb and 5441ed2631b8:

• pout and pmod now have 3 decimal places (instead of 1) in the pa_qc.dat file. (pout is set there, the netCDF writer uses the pout, tout, and hout rows for their corresponding ?mod variables.) However, pout will be limited by upstream accuracy in the runlogs. The runlogs do 2 decimal places, and for Park Falls at least it looks like the pressure is only recorded to 0.1 hPa.
• zmin was also a problem for the observation operator. It was set to 3 decimal places in the QC file, and only has 3 in the .col file, but has 5 in the .ray file. gfit has been updated to output 5 decimal places in the .col files, and the pa_qc.dat file now uses f9.5 as it's format. This was a compromise; gfit uses f10.5 now to ensure there's a space before it (max length should be sXX.XXXXX = 9 characters, s is the sign, zmin will never have 3 digits before the decimal, though it should really never have < -10 km, so the realistic max length is 8), but the netCDF writer expects the qc.dat file to have fixed width columns, and I think "f10.5" would break that. It only looks at the value after the decimal (the "5" in "f9.5"), so this shouldn't matter.
• xch4 and xch4_error: since we give these in ppm, the old "f9.4" and "f8.4" precisions could cause minor quantization in the values. Both have been increased to "f9.5".

Note: users will need to be instructed to update the precisions in their site's QC files. Alternatively, we could just not round values in the netCDF file, though it looks like only the .aia data is rounded; line 3478 is the only place I see the digit from the qc.dat file used:

digit = int(qc_data['format'][qc_id].split('.')[-1])
aia_qc_data = np.round(aia_data[var][start:end].values*qc_data['rsc'][qc_id],digit)

Searching for 'format' and "format" only show cases where it is written as an attribute.

[joshua-laughner]

Benchmark .mod and .vmr files were updated in Mercurial changeset d6d1e230de6a.