pydata / pydata/xarray

Comparison with masked array yields object-array with nans for masked values

Open
#1,792 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Code Sample, a copy-pastable example if possible
$ cat mwe.py 
#!/usr/bin/env python3.6
import xarray
import numpy

da = xarray.DataArray(numpy.arange(5))
ma = numpy.ma.masked_array(numpy.arange(5), [True, False, False, False, True])
print(da>ma)
$ ./mwe.py 
<xarray.DataArray (dim_0: 5)>
array([nan, False, False, False, nan], dtype=object)
Dimensions without coordinates: dim_0
Problem description

A comparison between a DataArray and a masked_array results in an array with dtype object instead of an array with dtype bool. This is problematic, because code should be able to assume that x > y returns something with a bool dtype.

Expected Output

I would expect the masked array to be dropped (which it is) and an array to be returned equivalent to the comparison da>ma.data

<xarray.DataArray (dim_0: 5)>
array([False, False, False, False, False], dtype=bool)
Dimensions without coordinates: dim_0
Output of xr.show_versions()
INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Linux OS-release: 2.6.32-696.6.3.el6.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8

xarray: 0.10.0+dev12.gf882a58
pandas: 0.21.0
numpy: 1.13.3
scipy: 1.0.0
netCDF4: 1.3.1
h5netcdf: None
Nio: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.16.0
matplotlib: 2.1.0
cartopy: None
seaborn: 0.8.1
setuptools: 38.2.4
pip: 9.0.1
conda: 4.3.16
pytest: 3.1.2
IPython: 6.1.0
sphinx: 1.6.2
None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied mwe.py example with the reported Python, NumPy, and xarray versions, then trace how DataArray comparisons handle a numpy masked_array. Add a regression test for the comparison result and verify that the output has bool dtype with masked values handled as shown in the expected output.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.