pydata / pydata/xarray

KeyError when selecting "nearest" data with given tolerance

Open
#4,995 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic-indexing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened: Selecting data with ds.sel(index=given_index, method="nearest", tolerance=tolerance) only works in case for each given_index exists an index that is within the given tolerance, otherwise a `KeyError: "not all values found in index 'index'" occurs.

What you expected to happen: I would expect, that those indices that are not within the tolerance would simply be dropped.

Minimal Complete Verifiable Example:

import xarray as xr
ds = xr.DataArray([1,2,3,4,5], dims=["lat"], coords={'lat':[10,20,30,50,60]})

# working example, all latitudes are within the tolerance
ds.sel(lat=[5,15,40], method="nearest", tolerance=10)
#<xarray.DataArray (lat: 3)>
#array([1, 2, 4])
#Coordinates:
#  * lat      (lat) int64 10 20 50

# failing example, latitude 40 is not within the tolerance
ds.sel(lat=[5,15,40], method="nearest", tolerance=5)
# KeyError: "not all values found in index 'lat'"

I would expect

>>> ds.sel(lat=[5,15,40], method="nearest", tolerance=5)
<xarray.DataArray (lat: 2)>
array([1, 2])
Coordinates:
  * lat      (lat) int64 10 20

Anything else we need to know?:

Environment:

Output of xr.show_versions()

INSTALLED VERSIONS

commit: 66acafa7f1f1477cfd6c5b7c3458859763433092
python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:12:38)
[Clang 11.0.1 ]
python-bits: 64
OS: Darwin
OS-release: 20.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.UTF-8
libhdf5: 1.10.6
libnetcdf: 4.7.4

xarray: 0.15.2.dev452+g66acafa7.d20210304
pandas: 1.2.3
numpy: 1.20.1
scipy: 1.6.0
netCDF4: 1.5.6
pydap: installed
h5netcdf: 0.10.0
h5py: 3.1.0
Nio: None
zarr: 2.6.1
cftime: 1.4.1
nc_time_axis: 1.2.0
PseudoNetCDF: installed
rasterio: 1.2.0
cfgrib: 0.9.8.5
iris: 2.4.0
bottleneck: 1.3.2
dask: 2021.02.0
distributed: 2021.02.0
matplotlib: 3.3.4
cartopy: 0.18.0
seaborn: 0.11.1
numbagg: installed
pint: 0.16.1
setuptools: 49.6.0.post20210108
pip: 20.2.4
conda: None
pytest: 6.2.2
IPython: None
sphinx: None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the DataArray.sel entry point and trace the nearest-indexing path used with tolerance. Reproduce the example as a regression test, then verify that out-of-tolerance labels are dropped while valid labels remain selected. Done means the failing example no longer raises KeyError and the expected two-item result is covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.