Python embedding breaks with MET V12

[caroline-owens]

I have a python embedding script that successfully reads dataplanes as a numpy array (dtype uint8) in MET v 11.0.2 (using the METplus tutorial docker image, has python 3.6.8).

In MET v 12.0.4 (using the METplus docker image, comes with python 3.10), the script no longer works. It fails with the following messages:

PYTHON: Module:        'met.dataplane'
PYTHON: User python command:   'read_model_grids.py /rp_data/models/HAB_LERIE/archive/2024070115/gtiff/2024070211.tif CIcyano'
Python Script:  'read_model_grids.py'
Input File:     '/rp_data/models/HAB_LERIE/archive/2024070115/gtiff/2024070211.tif'
Data Name:      'CIcyano'
Shape: (504, 1454)
Bounds: BoundingBox(left=-83.68329999781889, bottom=41.2746896872549, right=-78.615277965881, top=43.031418094474084)
Data Shape:     (504, 1454)
Data Type:      dtype('uint8')
Attributes:     {'valid': '20240702_110000', 'init': '20240701_150000', 'lead': '000020', 'accum': '000000', 'name': 'CIcyano', 'long_name': 'CIcyano_model', 'level': 'Surface', 'units': 'None', 'fill_value': 255, 'grid': {'type': 'LatLon', 'name': 'OLCI_L3', 'Nlat': 504, 'Nlon': 1454, 'lat_ll': 41.2746896872549, 'lon_ll': -83.68329999781889, 'delta_lat': 0.0034855722365459965, 'delta_lon': 0.0034855722365459965}}
 PYTHON: validating the dataplane array...
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/numpy/ma/core.py", line 467, in _check_fill_value
    fill_value = np.array(fill_value, copy=False, dtype=ndtype)
ValueError: cannot convert float NaN to integer

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/share/met/python/met/dataplane.py", line 313, in <module>
    main(sys.argv)
  File "/usr/local/share/met/python/met/dataplane.py", line 305, in main
    met_data = dataplane.validate_met_data(init_met_data, fill_value)
  File "/usr/local/share/met/python/met/dataplane.py", line 202, in validate_met_data
    met_data = np.ma.masked_equal(met_data, float('nan'))
  File "/usr/local/lib/python3.10/site-packages/numpy/ma/core.py", line 2108, in masked_equal
    output.fill_value = value
  File "/usr/local/lib/python3.10/site-packages/numpy/ma/core.py", line 3765, in fill_value
    target = _check_fill_value(value, self.dtype)
  File "/usr/local/lib/python3.10/site-packages/numpy/ma/core.py", line 473, in _check_fill_value
    raise TypeError(err_msg % (fill_value, ndtype)) from e
TypeError: Cannot convert fill_value nan to dtype uint8
WARNING: 
WARNING: straight_python_dataplane() -> an error occurred importing module "read_model_grids"
WARNING: 
ERROR  : 
ERROR  : Trouble reading data from forecast file "PYTHON_NUMPY"

Is uint8 no longer supported as an input format? My python embedding script is attached here (read_model_grids.py).

I'm noticing that even though I specified my fill_value is 255 in the attrs dict, the error messages are still suggesting the code may try to use nan as a fill_value?

Thank you!!

[caroline-owens]

I am not sure if this might be somehow related to the issue described here: dtcenter/MET#2557 although that seems to be closed/resolved .. thank you so much!

[caroline-owens]

I'm so sorry, I found the answer in the docs Appendix F 38.4.1 which clearly shows data needs to be double precision floating point. Thank you for all the incredible documentation and support!

[j-opatz]

I'm glad you were able to find the answer @caroline-owens,

And I hope the documentation continues to serve you well! Please let us know if there are any further questions or difficulties that you encounter.