Add capability to process AHI L2 products of cloud top pressure and c…

[xwzhang615]

Description

This PR adds the capability to read different file names corresponding with changes throughout the years, allowing to process BT , cloud mask, cloud top pressure and cloud type from Himawari-8/9 AHI L1 and L2 product datasets. The size of the array is hardcoded nband+3 in order to process cloud mask, cloud top pressure and cloud type together and write them out to the same file. These products need to be at the same resolution. The main modification files for this AHI processing are included in the himawari_ahi_mod.F90 and 'utils_mod.F90'.

[byoung-joo]

In src/utils_mod.F90, we have been using nband+1 for cloud mask or cloud fraction fields. This is assumed both for ABI and AHI.

This PR assumes nband+2 and nband+3 for other L2 products, but only for AHI.

I guess src/utils_mod.F90 needs to be more generic.

@xwzhang615, do you intend to write out the additional AHI L2 fields into IODA format ?

[xwzhang615]

In src/utils_mod.F90, we have been using nband+1 for cloud mask or cloud fraction fields. This is assumed both for ABI and AHI.

This PR assumes nband+2 and nband+3 for other L2 products, but only for AHI.

I guess src/utils_mod.F90 needs to be more generic.

@xwzhang615, do you intend to write out the additional AHI L2 fields into IODA format ?

Yes, this code has an ability to write out the additional AHI L2 products into the mpas_iodav1.nc file.

[byoung-joo]

Yes, this code has an ability to write out the additional AHI L2 products into the mpas_iodav1.nc file.

For ABI, it can write out additional L2 product to a MPAS format, but NOT to an IODA format.
As src/utils_mod.F90 is shared by both ABI and AHI, this part should be written in more generic way.

The question is.... should we merge this PR first, then generalize it in a separate PR? or can we make it generic in this PR ?

What do you think, @ibanos90 ?

[ibanos90]

Yes, this code has an ability to write out the additional AHI L2 products into the mpas_iodav1.nc file.

For ABI, it can write out additional L2 product to a MPAS format, but NOT to an IODA format. As src/utils_mod.F90 is shared by both ABI and AHI, this part should be written in more generic way.

The question is.... should we merge this PR first, then generalize it in a separate PR? or can we make it generic in this PR ?

What do you think, @ibanos90 ?

It's great to have the addition of these products to the code, so we can have similar capabilities between ABI and AHI. I wonder how the modifications in output_iodav1_o2m affect ABI. I think that if the changes do not break the conversion of ABI files, we can work on make it generic on a following PR. But, if they do crash for ABI, then we have to address that as we should ensure that current capabilities still work when merging new code. Does that make sense?
@xwzhang615 did you test both AHI and ABI?

[xwzhang615]

Yes, this code has an ability to write out the additional AHI L2 products into the mpas_iodav1.nc file.

For ABI, it can write out additional L2 product to a MPAS format, but NOT to an IODA format. As src/utils_mod.F90 is shared by both ABI and AHI, this part should be written in more generic way.
The question is.... should we merge this PR first, then generalize it in a separate PR? or can we make it generic in this PR ?
What do you think, @ibanos90 ?

It's great to have the addition of these products to the code, so we can have similar capabilities between ABI and AHI. I wonder how the modifications in output_iodav1_o2m affect ABI. I think that if the changes do not break the conversion of ABI files, we can work on make it generic on a following PR. But, if they do crash for ABI, then we have to address that as we should ensure that current capabilities still work when merging new code. Does that make sense? @xwzhang615 did you test both AHI and ABI?

Oh, yes, for AHI is fine. Meanwhile, I tested this code for ABI datasets just now. I found the obs2mpas.x currently can read the ABI datasets, interpolate and generate the index_xxxx.nc file successfully, but the final process of write out mpas_iodav1.nc was failed. So, I think if we want to use the code in more generic way, some codes in 'src/utils_mod.F90' need to be additionally modified, like BJ said, the added dimension of nband+2 and nband+3 for saving other L2 products, may only valid for the AHI data structure, not for ABI.

[byoung-joo]

Thanks for adding this useful feature, @xwzhang615.

From the discussion and testing, however, this PR can not be merged in a current form. This will break the use for ABI. Also, some ABI's L2 products does not meet the condition of this PR: These products need to be at the same resolution.. That is why we only write out the additional ABI L2 product to MPAS format, NOT to IODA format.

Users can still utilize the L2 product in MPAS format to co-locate the obs in IODA format via cellIndex@MetaData. We can discuss further off-line.