Merge fails when sparse Dataset has overlapping dimension values
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Sparse numpy arrays used in a merge operation seem to fail under certain coordinate settings. for example, this works perfectly:
import xarray as xr
import numpy as np
data_array1 = xr.DataArray(data,name='default',
dims=['source','receiver','time'],
coords={'source':['X.1'],
'receiver':['X.2'],
'time':time}).to_dataset()
data_array2 = xr.DataArray(data,name='default',
dims=['source','receiver','time'],
coords={'source':['X.2'],
'receiver':['X.1'],
'time':time}).to_dataset()
dataset1 = xr.merge([data_array1,data_array2])
But this raises an IndexError: Only indices with at most one iterable index are supported. from the sparse package:
import xarray as xr
import numpy as np
import sparse
data = sparse.COO.from_numpy(np.random.uniform(-1,1,(1,1,100)))
time = np.linspace(0,1,num=100)
data_array1 = xr.DataArray(data,name='default',
dims=['source','receiver','time'],
coords={'source':['X.1'],
'receiver':['X.2'],
'time':time}).to_dataset()
data_array2 = xr.DataArray(data,name='default',
dims=['source','receiver','time'],
coords={'source':['X.2'],
'receiver':['X.1'],
'time':time}).to_dataset()
dataset1 = xr.merge([data_array1,data_array2])
I have noticed this occurs when the merger would seem to add dimensions filled with nan values.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the sparse.COO reproducer in the issue and compare it with the dense NumPy case. Trace the xarray merge path where overlapping coordinates introduce dimensions filled with NaN values, then verify that merging the sparse datasets completes without the sparse indexing error and preserves the expected dimensions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100