pydata / pydata/xarray

`DataArray.drop_sel()` along a MultiIndex-ed dimension drops too much data

Open
#6,354 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-indexing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened?

Calling the drop_sel method on a DataArray object produces an incorrect result if the axis involved is indexed by a MultiIndex.

Assuming that dimension 'direction' is indexed by a MultiIndex with named levels 'azimuth' and 'elevation' one can attempt to drop a single item along this dimension in two ways:

  • call da.drop_sel(direction=(0.0, 0.0)), but this drops all items whose 'azimuth'==0, regardless of 'elevation'
  • call da.drop_sel(azimuth=0.0, elevation=0.0), but this results in an error ValueError: dimension 'azimuth' does not have coordinate labels
What did you expect to happen?

I expected that at least the first call would result in a correctly dropped item along dimension 'direction'. This expectation is based on the fact that calling .sel(direction=(0.0, 0.0)) works and returns the element where both 'azimuth' and 'elevation' are 0.0.

It would be convenient for the second call to also work, but the analogous call to .sel(azimuth=0.0, elevation=0.0) does not work either, so I guess this is unsupported.

Minimal Complete Verifiable Example
import numpy as np
import xarray as xr

# Create a DataArray with a MultiIndex-ed dimension:
da = xr.DataArray(
    np.reshape(np.arange(16), (4, 4)),
    dims = ['azimuth', 'elevation'],
    coords={
        'azimuth': range(4),
        'elevation': range(4),
    }).stack(direction=('azimuth', 'elevation'))

# Correctly selects a single item along 'direction'
da.sel(direction=(0, 0))

# Incorrectly removes all items along 'direction' where azimuth==0...
da.drop_sel(direction=(0, 0))

# ...but it should remove just one item, like this call:
da.drop_isel(direction=0)

# This fails with 'ValueError: dimension 'azimuth' does not have coordinate labels':
da.drop_sel(azimuth=0, elevation=0)
Relevant log output

No response

Anything else we need to know?

No response

Environment

INSTALLED VERSIONS

commit: None
python: 3.9.7 (default, Sep 16 2021, 13:09:58)
[GCC 7.5.0]
python-bits: 64
OS: Linux
OS-release: 5.10.0-8-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_DK.UTF-8
LOCALE: ('en_DK', 'UTF-8')
libhdf5: 1.10.6
libnetcdf: 4.8.1

xarray: 0.20.1
pandas: 1.4.1
numpy: 1.21.2
scipy: 1.7.3
netCDF4: 1.5.7
pydap: None
h5netcdf: 999
h5py: 2.10.0
Nio: None
zarr: None
cftime: 1.5.1.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: None
distributed: None
matplotlib: 3.5.1
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
setuptools: 58.0.4
pip: 21.2.4
conda: None
pytest: None
IPython: 8.1.1
sphinx: None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the DataArray.drop_sel() and drop_isel() behavior shown in the minimal reproducible example, comparing it with DataArray.sel() on the stacked 'direction' MultiIndex. Reproduce the issue with the provided xarray, pandas, and NumPy setup; done means dropping direction=(0, 0) removes only that item while preserving the other entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.