ndarray._tolerant_array changes array coordinates even if they are identical to start with
- Dominant language
- Python
- Stars
- 7
- Forks
- 10
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 5
Description
# 🐜 Bug Report
Probably a bug, or at least undesireable behaviour
When routines call [`ndarray._tolerant_array`](https://github.com/MetOffice/ANTS/blob/811bd5538234153b6483c3e725b0d1972a7e4769/lib/ants/utils/ndarray.py#L214) with two arrays (in terms of coordinate values and steps) via the ndarray.merge_array routine, the result is a potentially different set of output arrays due to rounding to `ants.config.TOLERANCE` precision.
This is definitely undesireable in some cases but could be needed in others. We should investigate changing this behaviour/or calls to it and pass changes on to workflow owners for scientific assessment with real data and domains.
## How to reproduce
Step by step guide to reproduce the behaviour:
1. Feed two arrays with overlapping coordinates with identical step sizes and values in the overlap with precision greater than ants.config.TOLERANCE into ndarray.merge_array
2. Inspect the resulting array coordinates
3. No original values will be present
## Environment
I am using the ANTS 2.2.0 env (and earlier 2.x series)
## Version
This bug exists in ANTS 2.2.0 (and earlier)
## Additional Context
This bug was first encountered via the merge routine, which calls the `_unified_grid` routine, which calls `ndarray.merge_array` to combine the coordinates of the two cubes being merged. Within the `merge_array` routine, `_tolerant_array` is being called which rounds coordinate values to the `ants.config.TOLERANCE`. In practice this can mean that in pipelines where datasets have to be merged, coordinates can end up slightly misaligned vs other ancillary files being generated for the same domain.
Contributor guide
Research direction
Start in lib/ants/utils/ndarray.py at _tolerant_array, then trace how _unified_grid and merge_array invoke it. Reproduce the reported case with overlapping coordinates whose precision exceeds ants.config.TOLERANCE, and compare the resulting coordinates with the inputs. Done means the desired preservation or tolerance behavior is established and validated with relevant workflow owners and real data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100