MetOffice / MetOffice/lfric_core
Honour `_FillValue` attribute when loading ancillary files
Nobody has claimed this yet.
- Dominant language
- Fortran
- Stars
- 26
- Forks
- 73
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 4
Description
### Are there any linked Issues or Pull Requests?
- ticket:4168 - Original LFRic Trac ticket.
- [ancil:2130](https://code.metoffice.gov.uk/trac/ancil/ticket/2130) - ANCIL workaround Trac ticket.
### Brief description
Presently, the `_FillValue` attribute is ignored when LFRic reads in ancillary files (and presumably other input files). This attribute ([attribute conventions](https://docs.unidata.ucar.edu/nug/current/attribute_conventions.html)) is used to identify missing data] (e.g. to mask out land points for sea-only fields).
### Further details of the issue.
Because the `_FillValue` attribute is ignored, this imposes a constraint on any tools creating files to be read into LFRic that they need to use the model RMDI value as a `_FillValue`.
Instead, the LFRic load should recognise `_FillValue` as representing missing data and translate this to RMDI.
There is an additional complexity in that the netCDF specification (linked above) includes "It is not necessary to define your own _FillValue attribute for a variable if the default fill value for the type of the variable is adequate". So the absence of a `_FillValue` attribute does not imply there is no missing data.
The default fill values can be found most easily with python:
```python
In [1]: import netCDF4
In [2]: netCDF4.default_fillvals
Out[2]:
{'S1': '\x00',
'i1': -127,
'u1': 255,
'i2': -32767,
'u2': 65535,
'i4': -2147483647,
'u4': 4294967295,
'i8': -9223372036854775806,
'u8': 18446744073709551614,
'f4': 9.969209968386869e+36,
'f8': 9.969209968386869e+36}
```
Note that none of these defaults match the LFRic RMDI (the largest finite negative value).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No implementation files, tests, or entry points are named. Start by locating the LFRic ancillary-file loader and its netCDF attribute handling, then inspect how missing values become RMDI. Done means explicit and type-default _FillValue values are recognised and translated to RMDI, with coverage for the supported input types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fortran
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100