numpy / numpy/numpy

BUG (Possible): masked array divide by zero array seems to screen out nan and inf

Open
#18,744 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component: numpy.ma
Dominant language
Python
Stars
32.8k
Forks
12.8k
Avg merge
1d 7h
Merged PRs (30d)
197

Description

Under certain circumstances dividing a masked array by regular array with zeros seems to unexpactantly screen out nan and inf answers.

Reproducing code example:
import numpy as np
from numpy import ma

# Make masked and regular array
x = np.array([ 0.,  1., 0.,  1.])
xm = ma.masked_equal(x, -1)
y = np.array([ 0.,  0., 0.,  0.])

If we divide x by y we get:

x/y
Out[239]: array([nan, inf, nan, inf])

If we divide xm by y we get:

xm/y
Out[240]: 
masked_array(data=[--, --, --, --],
             mask=[ True,  True,  True,  True],
       fill_value=-1.0,
            dtype=float64)

...it has masked the nan and inf values even though they are not -1

If we divide xm by y and get just the data we get:

(xm/y).data
Out[242]: array([0., 1., 0., 1.])

🤯.. now we have data where I would expect nan and inf. I'm not very experienced in Python but this looks like an unexpected result and I thought I should report it (I spent alot of time tracing unexpected results from a function that turns out to be due to this).

Edit: Where I found this in the wild is even more insidious because there was no specific .data step - it happened silently as follows:

Suppose our calculation was done in a function:

def somefunc3(a,b):
    c = a / b
    return c

somefunc3(xm, y)
Out[76]: 
masked_array(data=[--, --, --, --],
             mask=[ True,  True,  True,  True],
       fill_value=-1.0,
            dtype=float64)

It output the masked array without nan and inf.

Now suppose we were stuffing the result of somefunc into a larger array:

d = np.zeros((4, 2))

d[:,0] = somefunc3(xm, y)
d
Out[77]: 
array([[0., 0.],
       [1., 0.],
       [0., 0.],
       [1., 0.]])

Now it silently converted the masked array back to a regular array and put in 1 or 0 when it should be nan or inf. Note that when I ran this on my machine I got a divide by zero warning only one time, but all other times I ran it I did not (I have no idea why).

NumPy/Python version information:

1.18.1 3.7.6 (default, Jan 8 2020, 13:42:34)
[Clang 4.0.1 (tags/RELEASE_401/final)]
Edit: same behaviour on my other machine with versions:
1.19.2 3.8.5
[Clang 10.0.0]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the masked-array reproducer in the issue and compare division results with regular arrays, including assignment into a larger array. Trace the masked-array division behavior and determine the intended handling of nan and inf. Done means the behavior is clarified and covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.