MetOffice / MetOffice/ANTS

ndarray._tolerant_array changes array coordinates even if they are identical to start with

Open
#82 0 comments 0 reactions 0 assignees View on GitHub
➡ medium priority 🐜 bug
Dominant language
Python
Stars
7
Forks
10
Avg merge
1d 19h
Merged PRs (30d)
5

Description

# 🐜 Bug Report

Probably a bug, or at least undesireable behaviour

When routines call [`ndarray._tolerant_array`](https://github.com/MetOffice/ANTS/blob/811bd5538234153b6483c3e725b0d1972a7e4769/lib/ants/utils/ndarray.py#L214) with two arrays (in terms of coordinate values and steps) via the ndarray.merge_array routine, the result is a potentially different set of output arrays due to rounding to `ants.config.TOLERANCE` precision.

This is definitely undesireable in some cases but could be needed in others. We should investigate changing this behaviour/or calls to it and pass changes on to workflow owners for scientific assessment with real data and domains.

## How to reproduce
Step by step guide to reproduce the behaviour:

1. Feed two arrays with overlapping coordinates with identical step sizes and values in the overlap with precision greater than ants.config.TOLERANCE into ndarray.merge_array
2. Inspect the resulting array coordinates
3. No original values will be present

## Environment
I am using the ANTS 2.2.0 env (and earlier 2.x series)

## Version
This bug exists in ANTS 2.2.0 (and earlier)

## Additional Context

This bug was first encountered via the merge routine, which calls the `_unified_grid` routine, which calls `ndarray.merge_array` to combine the coordinates of the two cubes being merged. Within the `merge_array` routine, `_tolerant_array` is being called which rounds coordinate values to the `ants.config.TOLERANCE`. In practice this can mean that in pipelines where datasets have to be merged, coordinates can end up slightly misaligned vs other ancillary files being generated for the same domain.

Contributor guide

Open the contributing guide

Research direction

Start in lib/ants/utils/ndarray.py at _tolerant_array, then trace how _unified_grid and merge_array invoke it. Reproduce the reported case with overlapping coordinates whose precision exceeds ants.config.TOLERANCE, and compare the resulting coordinates with the inputs. Done means the desired preservation or tolerance behavior is established and validated with relevant workflow owners and real data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.