Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Framework doesn't handle datasets with axes defined as variables with 2-D coordinates without "axis" attributes #282

Open
wrongkindofdoctor opened this issue Oct 14, 2021 · 0 comments
Assignees
Labels
framework Issue pertains to the framework code

Comments

@wrongkindofdoctor
Copy link
Collaborator

wrongkindofdoctor commented Oct 14, 2021

The mixed-layer depth (mld) POD SAM0-UNICON test model dataset contains i,j coordinates used to define the 2-D latitude and longitude arrays and required variables. The latitude and longitude are defined as variables, but not dimensions, in the file metadata. There are also no units associated with latitude or longitude (which is not the case for the similarly-defined lat and lon coordinates in the tropical_pacific_sea_level POD dataset referenced in issue #198 ).

netcdf SAM0-UNICON_r1i1p1f1_gn.so.mon {
dimensions:
	time = UNLIMITED ; // (60 currently)
	bnds = 2 ;
	lev = 60 ;
	j = 384 ;
	i = 320 ;
	vertices = 4 ;
variables:
	double time(time) ;
		time:bounds = "time_bnds" ;
		time:units = "days since 1850-01-01 00:00:00" ;
		time:calendar = "noleap" ;
		time:axis = "T" ;
		time:long_name = "time" ;
		time:standard_name = "time" ;
	double time_bnds(time, bnds) ;
	double lev(lev) ;
		lev:bounds = "lev_bnds" ;
		lev:units = "m" ;
		lev:axis = "Z" ;
		lev:positive = "down" ;
		lev:long_name = "ocean depth coordinate" ;
		lev:standard_name = "depth" ;
	double lev_bnds(lev, bnds) ;
	int j(j) ;
		j:units = "1" ;
		j:long_name = "cell index along second dimension" ;
	int i(i) ;
		i:units = "1" ;
		i:long_name = "cell index along first dimension" ;
	double latitude(j, i) ;
		latitude:standard_name = "latitude" ;
		latitude:long_name = "latitude" ;
		latitude:units = "degrees_north" ;
		latitude:missing_value = 1.e+20 ;
		latitude:_FillValue = 1.e+20 ;
		latitude:bounds = "vertices_latitude" ;
	double longitude(j, i) ;
		longitude:standard_name = "longitude" ;
		longitude:long_name = "longitude" ;
		longitude:units = "degrees_east" ;
		longitude:missing_value = 1.e+20 ;
		longitude:_FillValue = 1.e+20 ;
		longitude:bounds = "vertices_longitude" ;
	double vertices_latitude(j, i, vertices) ;
		vertices_latitude:units = "degrees_north" ;
		vertices_latitude:missing_value = 1.e+20 ;
		vertices_latitude:_FillValue = 1.e+20 ;
	double vertices_longitude(j, i, vertices) ;
		vertices_longitude:units = "degrees_east" ;
		vertices_longitude:missing_value = 1.e+20 ;
		vertices_longitude:_FillValue = 1.e+20 ;
	float so(time, lev, j, i) ;
		so:standard_name = "sea_water_salinity" ;
		so:long_name = "Sea Water Salinity" ;
		so:comment = "Sea water salinity is the salt content of sea water, often on the Practical Salinity Scale of 1978. However, the unqualified term \'salinity\' is generic and does not necessarily imply any particular method of calculation. The units of salinity are dimensionless and the units attribute should normally be given as 1e-3 or 0.001 i.e. parts per thousand." ;
		so:units = "0.001" ;
		so:history = "rewrote by cmor via python script 2019-04-03T14:38:47Z altered by CMOR: Converted units from \'gram/kilogram\' to \'0.001\'. 2019-04-03T14:38:47Z altered by CMOR: replaced missing value flag (9.96921e+33) with standard missing value (1e+20)." ;
		so:original_units = "gram/kilogram" ;
		so:cell_methods = "area: mean where sea time: mean (interval: 1 month)" ;
		so:cell_measures = "area: areacello volume: volcello" ;
		so:missing_value = 1.e+20f ;
		so:_FillValue = 1.e+20f ;
		so:coordinates = "latitude longitude" ;

Running using the CMIP standard, the framework preprocessor assumes that all dimension variables have "axis" attributes, and that all dimension variables are also defined as coordinates. The missing units cause an error in reconcile_scalar_coords that can be resolved by adding a check for a units attribute in the dataset (ds_scalars) array when defining the ds_axes array. If no units attribute exists, ds_axes is simply an empty array:

       ds_axes = []
       ds_axes = [c.axis for c in ds_scalars if "axis" in c.attrs]

A subsequent error occurs when converting the coordinate dimensions and bounds in the translated variable structure in units.convert_dataarray. The dimensions (dim_axes) are defined as time, lev, lat, and lon in the mld POD settings.jsonc file. However, the dim_axes are checked against the dataset (ds) structure defined using the file metadata and dimensions, which do not contain lat or lon, and thus trigger subsequent ValueError (where da_name is 'lat' or 'lon'):

da = ds.get(da_name, None) 
if da is None:
        raise ValueError(f"convert_dataarray: '{da_name}' not found in dataset.")
@wrongkindofdoctor wrongkindofdoctor added the framework Issue pertains to the framework code label Oct 14, 2021
@wrongkindofdoctor wrongkindofdoctor self-assigned this Oct 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
framework Issue pertains to the framework code
Projects
None yet
Development

No branches or pull requests

1 participant