Control metadata

[pnorton-usgs]

@jmccreight After meeting last week I wanted to document what we discussed and some of my thoughts. Hopefully we can use this to discuss and document possible changes to the control metadata.

Control

Legacy xml tags and attributes

Metadata was initially created from the published control variable tables and scraping the PRMS5 code.

Example of xml structure

<control>
    <control_param name="basinOutON_OFF" version="5.0" deprecated="6.0">
        <default>0</default>
				<force_default>0</force_default>
        <type>1</type>
        <numvals>1</numvals>
        <desc>Switch to specify whether or not basin summary output files are generated</desc>
        <values type="flag">
            <value name="0">off</value>
            <value name="1">on</value>
        </values>
        <related_variables>
            <variable name="basinOutVars"/>
            <variable name="basinOut_freq"/>
        </related_variables>
    </control_param>
	  ...
</control>

Each control variable has:

Key	Description	XML Element	XML Attribute	Allowed values
name	Name of the control variable	control_param	name
version	Version of PRMS where variable was introduced	control_param	version
deprecated	Optional version of PRMS when variable no longer supported	control_param	deprecated
default	Default value to use when one has not been specified	default
force_default	Optional boolean indicating the default value should always be used	force_default		0, 1
type	Data type of the control variable values	type		1, 2, 3, 4
numvals	Number of values allowed for variable	numvals
desc	Description of variable	desc
values	Optional block for denoting accepted values of variable	values
values->type	Type of valid values	values	type	flag, module, interval, method, parameter
value	Description of valid value in values block	value
value->name	A valid value	value	name
value->version	Version when a value option was added	value	version
related_variables	A group of variables that are associated with current variable	related_values
variable->name	Name of control variable associated with current variable	variable	name

type description

1: int32
2: float32
3: float64
4: string

values->type description

flag - Either boolean compatible values (1/0 = true/false, on/off, yes/no) or multiple values indicating behavior(s) to trigger
interval - values indicating the time-series interval to use (e.g. 0 = daily)
method - values indicating an action to perform or module to act on
module - values representing modules to use in simulation code
parameter - values denoting one or more parameters to effect

Challenges

• no way to programmatically figure out if a control variable value is a scalar or list
  • could create context key which would have values of scalar, array, etc
    • remove numvals?
• version information is currently incomplete in the XML file
• how to handle start_time and end_time which are really datetime values but stored in control file as an array of integers

pynhm

Legacy XML structures are converted to a dictionary and written to YAML.

YAML example

basinOutON_OFF:
  default: 0
  desc: Switch to specify whether or not basin summary output files are generated
    (0=no; 1=yes)
  numvals: 1
  related_variables:
  - basinOutVars
  - basinOut_freq
  type: 1
  values:
    0: 'off'
    1: 'on'
  values_type: flag

Suggested changes

• missing keys
  • version, deprecated, force_default
• version usage could be changed to also indicate a range of versions. If we did this then the deprecated key would not be necessary.
  • e.g. version=">=5.0.0 & <5.2.0"
• add context key with possible values scalar, array, etc
• suggested name changes
  • type -> datatype
  • values -> valid_values
  • values_type -> valid_values_type
• change datatype (aka type) from integer values to strings:

   int32: 1
   float32: 2
   float64: 3
   string: 4
   datetime: