Proposed variable_id suffixes for uncertainty information - feedback requested

Our discussions with experts from the observations community during the Hackathon as well as follow-up discussions with the obs4MIPs panel have led us to come up with the following plan for the treatment of uncertainties within the REF:

- Each additional uncertainty information is provided in a separate file. (We explored the idea of including all uncertainty information as ancillary variables int he same file, but this is likely not feasible due to an infrastructure -related issue at this time.)
- Netcdf file containining the main geophysical variable has global attribute “has_auxdata”, and is set to TRUE when additional files containing uncertainty information are provided. It should be set to FALSE when no uncertainty information is provided. If this attribute does not exist, the REF will assume there is no uncertainty information provided.
- If has_auxdata is set to TRUE, netcdf file containing the main geohpysical variable has global attribute “aux_variable_id”. This contains all additional variable_ids for the uncertainty information provided, in the form of a string with spaces as delimiters.
- Global attribute “variable_id” of file containing the uncertainty information corresponds to variable name+”_”+accepted suffix. For accepted suffixes, see Table 1.
- Variable name within the netcdf file corresponds to variable_id.
- Technical note provides detailed explanation of each additional uncertainty information provided, including type of uncertainty. The note should discuss what is included in the total uncertainty, by providing a breakdown of its components; such as sources of independent, structured/correlated and common uncertainty.

Table 1: Required suffixes by the REF
Suffix | Long name | Description
-- | -- | --
stderr | Standard error | The standard error on the mean. The standard error includes both systematic and statistical uncertainty. By default it is assumed that the values supplied are for one standard error.
stddev | Standard deviation | Alternative to stderr for retrieval quantities that are binary or fractional.
nobs | Number of observations | The number of discrete observations or measurements from which a data value has been derived. The use of this modifier is deprecated and the standard_name number_of_observations is preferred to describe this type of metadata variable.
lbnd | Lower bound | Alternative for observations with asymmetrical uncertainty distribution, lower bound of the uncertainty.
ubnd | Upper bound | Alternative for observations with asymmetrical uncertainty distribution, upper bound of the uncertainty.

Table 2: Desired suffixes, which might be out of scope within the REF timeframe
Suffix | Long name | Description
-- | -- | --
tunc | Total uncertainty | The total per-datum uncertainty associated with the geophysical variable.  If the independent, structured and common uncertainty components are also provided, this would be equal to the sum in quadrature of these components.
iunc | Individual uncertainty | The per-datum component of uncertainty that is uncorrelated between observations.
sunc | Structured/correlated uncertainty | The per-datum component of uncertainty that is structured and correlated over a defined space/time scale.  This correlation length scale in space and time must be provided in the variable metadata.
cunc | Common uncertainty | The per-datum component of uncertainty that is common to all observations of a given type (specific instrument, resolution, variable type).

**Please note these two tables are based on our initial lists. We may need to reduce them in the short term to fit with the infrastructure.** However, we decided to share these lists to ask for feedback from both the observations communities as well as the modelling communities (specifically the developers of CMOR).