pydata / pydata/xarray

assert_equal does not handle wrapped duck arrays well

Open
#5,570 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic-arrays topic-testing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Whilst trying to fix #5559 I noticed that xarray.testing.assert_equal (and xarray.testing.assert_equal) don't behave well with wrapped duck-typed arrays.

Firstly, they can give unhelpful AssertionError messages:

In [5]: a = np.array([1,2,3])

In [6]: q = pint.Quantity([1,2,3], units='m')

In [7]: da_np = xr.DataArray(a, dims='x')

In [8]: da_p = xr.DataArray(q, dims='x')

In [9]: da_np
Out[9]: 
<xarray.DataArray (x: 3)>
array([1, 2, 3])
Dimensions without coordinates: x

In [10]: da_p
Out[10]: 
<xarray.DataArray (x: 3)>
<Quantity([1 2 3], 'meter')>
Dimensions without coordinates: x

In [11]: from xarray.testing import assert_equal

In [12]: assert_equal(da_np, da_p)
/home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/xarray/core/duck_array_ops.py:265: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray.
  flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2))
/home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/xarray/core/duck_array_ops.py:265: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
  flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2))
/home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/xarray/core/duck_array_ops.py:265: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray.
  flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2))
/home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/xarray/core/duck_array_ops.py:265: DeprecationWarning: elementwise comparison failed; this will raise an error in the future.
  flag_array = (arr1 == arr2) | (isnull(arr1) & isnull(arr2))
/home/tegn500/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/numpy/core/_asarray.py:102: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray.
  return array(a, dtype, copy=False, order=order)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-12-33b16d6b79ed> in <module>
----> 1 assert_equal(da_np, da_p)

    [... skipping hidden 1 frame]

~/miniconda3/envs/py38-mamba/lib/python3.8/site-packages/xarray/testing.py in assert_equal(a, b)
     79     assert type(a) == type(b)
     80     if isinstance(a, (Variable, DataArray)):
---> 81         assert a.equals(b), formatting.diff_array_repr(a, b, "equals")
     82     elif isinstance(a, Dataset):
     83         assert a.equals(b), formatting.diff_dataset_repr(a, b, "equals")

AssertionError: Left and right DataArray objects are not equal

Differing values:
L
    array([1, 2, 3])
R
    array([1, 2, 3])

These are different, but not because the array values are different. At the moment .values is converting the wrapped array type by stripping the units too - it might be better to check the type of the wrapped array first, then use .values to compare. Or could we even do duck-typed testing by delegating via expected.data.equals(actual.data)? (EDIT: I don't think a .equals() method exists in the numpy API, but you could do the equivalent of assert all(expected.data == actual.data)

Secondly, given that we coerce before comparison, I think it's possible that assert_equal could say two different wrapped duck-type arrays are equal when they are not, just because np.asarray() coerces them to the same values.

EDIT2: Looks like there is some discussion here

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in xarray/testing.py, where assert_equal compares DataArray objects, and inspect xarray/core/duck_array_ops.py for the coercion and comparison behavior shown in the example. Reproduce the pint.Quantity case, then determine how wrapped duck arrays should be compared without losing useful failure details or treating distinct arrays as equal. Done means assertions handle these cases correctly with focused regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.