Read in the units for variables on the stream files
- Dominant language
- Fortran
- Stars
- 32
- Forks
- 68
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 2
Description
Right now the stream infrastructure only reads in the units for the vertical coordinate (if it exists), and for time. The units for variables is ignored. This is sometimes important because the units may be different from what's expected and does change over time.
## Fragile code bits that would be made robust with reading the files:
Because this is lacking there are some hacks in the code with various ways to figure out what the units are. For example see these code bits...
### Checking for units in the Fortran files
``` shell
git grep units | grep \.F90
datm/datm_datamode_clmncep_mod.F90: real(r8) :: tbotmax ! units detector
datm/datm_datamode_clmncep_mod.F90: real(r8) :: tdewmax ! units detector
datm/datm_datamode_era5_mod.F90: real(r8) :: t2max ! units detector
datm/datm_datamode_era5_mod.F90: real(r8) :: td2max ! units detector
datm/datm_datamode_gefs_mod.F90: real(r8) :: tbotmax ! units detector
datm/datm_datamode_gefs_mod.F90: real(r8) :: maskmax ! units detector
datm/datm_pres_ndep_mod.F90: ! convert ndep flux to units of kgN/m2/s (input is in gN/m2/s)
docn/docn_datamode_cplhist_mod.F90: logical :: units_CToK = .true. ! true => convert SST in C to K
docn/docn_datamode_cplhist_mod.F90: ! use existing nml variable sst_constant_value to signal units of input
docn/docn_datamode_cplhist_mod.F90: units_CToK = .false. !in K already, don't convert
docn/docn_datamode_cplhist_mod.F90: if (units_CToK) then
```
### DATM, TBOT units detection (K or C):
The determination of if TBOT is in Celcius or Kelvin is if it's less than or equal to zero over the domain. This is pretty robust over a global domain -- but could easily fail for a single point or regional grid.
### DATM, TDEW units detection (K or C):
The determination of tdewmax for TDEW in Celcius or Kelvin is if the max over the entire grid is less than 50 assume Celcius. This seems less likely to fail for a smaller grid, but isn't entirely robust either.
### DATM, mask units detection:
The determination of maskmax doesn't seem to be actually used.
### DOCN, SST units detection (K or C):
In DOCN the units conversion is maybe more robust. But, it's gotten from the input namelist variable sst_constant_value and if it's greater than 230 it'll assume they are in Kelvin. But, that means that value has to be aligned with the datafile which isn't neccessarily guaranteed if the user makes an error.
### DATM Checking for units of NDEP fields:
The units for NDEP changed between CMIP6 (gN/m2/s) and CMIP7 (kgN/m2/s). So the units need to be converted. It's hard to tell by the data size what the units are, because they are always small. So what's being done is to assume it's CMIP6 units if only two fields are input (NHX, and NOY) and CMIP7 format if 4 fields are input (NHX and NOY split each into dry and wet).
For CESM3 era CPLHIST files the units are CMIP7 units, but the files provide both the 2 fields AND the 4 fields broken down. Hence, the user can easily make a mistake and provide the wrong list.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the stream infrastructure that currently reads vertical-coordinate and time units, then inspect the listed DATM and DOCN Fortran modules for the existing unit-detection workarounds. Trace how variable metadata is passed into the data models. Done means variable units are read from stream files and the affected conversions no longer depend on value heuristics or field-count assumptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fortran
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100