DataArray.drop_isel / .drop_sel with duplicated initial time stamp - InvalidIndexError
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What is your issue?
I have a DataArray ds with 7098 time steps in cftime format. One of these time steps is a duplicate (pos: 7087 & 7088).
I tried to drop the first instance using both ds.drop_isel(time=[7087]) and ds.drop_sel(ds.time[7087]) but get the error:
pandas.errors.InvalidIndexError: Reindexing only valid with uniquely valued Index objects.
The method works if I do not have the duplicate in it. Maybe this is the wrong method in the first place but it seemed to be the way to go. I am confused on how to get rid of the duplicate, and, if the method is the correct method, if this might be an issue.
ds
Out[247]:
<xarray.DataArray 'LST_Night_CMG' (time: 3, y: 2, x: 2)>
array([[[14357., 14357.],
[14342., 14358.]],
[[14409., 14409.],
[14435., 14388.]],
[[14409., 14409.],
[14435., 14388.]]])
Coordinates:
* time (time) object 2019-09-18 00:00:00 ... 2019-09-19 00:00:00
* x (x) float64 67.98 68.03
* y (y) float64 40.12 40.07
spatial_ref int32 0
Attributes:
coordinates: spatial_ref band
scale_factor: 1.0
add_offset: 0.0
ds.time
Out[249]:
<xarray.DataArray 'time' (time: 3)>
array([cftime.DatetimeProlepticGregorian(2019, 9, 18, 0, 0, 0, 0, has_year_zero=True),
cftime.DatetimeProlepticGregorian(2019, 9, 19, 0, 0, 0, 0, has_year_zero=True),
cftime.DatetimeProlepticGregorian(2019, 9, 19, 0, 0, 0, 0, has_year_zero=True)],
dtype=object)
Coordinates:
* time (time) object 2019-09-18 00:00:00 ... 2019-09-19 00:00:00
spatial_ref int32 0
ds.drop_isel(time=1)
Traceback (most recent call last):
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\IPython\core\interactiveshell.py", line 3444, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-248-0f89b7589e90>", line 1, in <module>
ds.drop_isel(time=1)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\dataarray.py", line 2413, in drop_isel
dataset = dataset.drop_isel(indexers=indexers, **indexers_kwargs)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\dataset.py", line 4565, in drop_isel
ds = ds.loc[dimension_index]
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\dataset.py", line 563, in __getitem__
return self.dataset.sel(key)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\dataset.py", line 2505, in sel
self, indexers=indexers, method=method, tolerance=tolerance
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\coordinates.py", line 422, in remap_label_indexers
obj, v_indexers, method=method, tolerance=tolerance
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\indexing.py", line 120, in remap_label_indexers
idxr, new_idx = index.query(labels, method=method, tolerance=tolerance)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\indexes.py", line 240, in query
indexer = get_indexer_nd(self.index, label, method, tolerance)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\xarray\core\indexes.py", line 142, in get_indexer_nd
flat_indexer = index.get_indexer(flat_labels, method=method, tolerance=tolerance)
File "C:\Users\einfa\anaconda3\envs\climate\lib\site-packages\pandas\core\indexes\base.py", line 3442, in get_indexer
raise InvalidIndexError(self._requires_unique_msg)
pandas.errors.InvalidIndexError: Reindexing only valid with uniquely valued Index objects
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the duplicated cftime coordinate, then trace drop_isel through xarray/core/dataarray.py and xarray/core/dataset.py into the indexing code shown in the traceback. Check existing tests for drop_isel or duplicate indexes and add coverage for removing one positional duplicate. Done means the reported operation behaves consistently or the supported workaround is documented by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100