numpy / numpy/numpy

BUG: MaskedArray does not seem to respect ufunc dispatch hierarchy

Open
#15,200 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

component: numpy.ma
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

When using a custom array container that implements __array_ufunc__ and is meant to be able to wrap MaskedArrays, non-commutativity occurs in binary operations with MaskedArrays. Despite what the below comment mentions:

https://github.com/numpy/numpy/blob/v1.17.3/numpy/ma/core.py#L3960-L3972

it does not properly defer, as seen in the example below. I am unfortunately not well-acquainted with the MaskedArray internals, so I don't know what would be a good way forward for a fix.

xref https://github.com/hgrecco/pint/issues/633

Reproducing code example:
import numpy as np
import numpy.lib.mixins


class WrappedArray(numpy.lib.mixins.NDArrayOperatorsMixin):
    __array_priority__ = 20

    def __init__(self, array, **attrs):
        self._array = array
        self.attrs = attrs

    def __repr__(self):
        return f"{self.__class__.__name__}(\n{self._array}\n{self.attrs}\n)"

    def __array__(self):
        return np.asarray(self._array)

    def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
        if method == '__call__':
            inputs = [arg._array if isinstance(arg, self.__class__) else arg
                      for arg in inputs]
            return self.__class__(ufunc(*inputs, **kwargs), **self.attrs)
        else:
            return NotImplemented


# Show basic wrap
w = WrappedArray(np.arange(3), test=1)
print(w)
print()

# Wrapping MaskedArrays works fine
wm = WrappedArray(np.ma.masked_array([1, 3, 5], mask=[False, True, False]),
                  test=2)
print(wm)
print()

# Operations with ndarrays work fine
a = np.array([2, 0, 1])
print(a * w)
print()

# Operations with masked arrays are not commutative
m = np.ma.masked_array([2, 0, 1], mask=[False, True, False])

# Good
print(w * m)
print()

# Bad
print(m * w)

Output:

WrappedArray(
[0 1 2]
{'test': 1}
)

WrappedArray(
[1 -- 5]
{'test': 2}
)

WrappedArray(
[0 0 2]
{'test': 1}
)

WrappedArray(
[0 -- 2]
{'test': 1}
)

[0 -- 2]
Numpy/Python version information:

1.17.3 3.6.7 | packaged by conda-forge | (default, Nov 6 2019, 16:19:42)
[GCC 7.3.0]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied Python reproducer and inspect numpy/ma/core.py around lines 3960-3972, then read the linked NEP-13 type-casting hierarchy discussion. Compare both operand orders involving a custom array_ufunc container and a MaskedArray. Done means the dispatch behavior is consistent in both orders and the relevant regression coverage passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.