Unsafe casting in netCDF save
- Dominant language
- Python
- Stars
- 7
- Forks
- 10
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 5
Description
# 🐜 Bug Report
The dtype casting for the netCDF4-classic save is incorrect. For example, `uint32` and `uint64` are being cast to `i8` (i.e. long int, `int64`), but netCDF4-classic does not permit long integers.
The list of permitted data types can be found [here](https://docs.unidata.ucar.edu/nug/current/md_types.html). Note the remark:
All the unsigned ints and the 64-bit ints are for CDF5 or netCDF-4 files only.
I suggest the best approach is to not attempt to cast as part of ANTS, but leave it to the underlying iris/netCDF libraries where possible. The only case where casting should be done is for boolean data, where the data should be cast to `i1` (i.e. "byte") with a valid_range attribute of `[0, 1]` added. Any other dtype casting should be the responsibility of the science application, not core ANTS code.
## Version
This bug exists in ANTS 2.2
## Additional Context
Numpy provides a `can_cast` method that is used in netCDF saving in ANTS:
This is generating a FutureWarning from dask:
FutureWarning: The numpy.can_cast function is not implemented by Dask array. You may want to use the da.map_blocks function or something similar to silence this warning. Your code may stop working in a future release.
Contributor guide
Research direction
Start by locating ANTS's netCDF4-classic save entry point and the numpy.can_cast usage mentioned in the report. Check how unsigned and 64-bit dtypes are handled, then verify that only booleans are cast to i1 with a valid_range of [0, 1] while other casting is left to the iris/netCDF libraries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100