Error concatenating Multiindex variables
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
MCVE Code Sample
>>> import xarray as xr
>>> da = xr.DataArray([0, 1], dims=["location"], coords={"lat": ("location", [10, 11]), "lon": ("location", [20, 21])}).set_index(location=["lat", "lon"])
>>> da2 = xr.DataArray([2, 3], dims=["location"], coords={"lat": ("location", [12, 13]), "lon": ("location", [22, 23])}).set_index(location=["lat", "lon"])
>>> xr.concat([da["location"], da2["location"]], dim="location")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/harry/code/xarray/xarray/core/concat.py", line 135, in concat
return f(objs, dim, data_vars, coords, compat, positions, fill_value, join)
File "/home/harry/code/xarray/xarray/core/concat.py", line 431, in _dataarray_concat
ds = _dataset_concat(
File "/home/harry/code/xarray/xarray/core/concat.py", line 384, in _dataset_concat
result = Dataset(result_vars, attrs=result_attrs)
File "/home/harry/code/xarray/xarray/core/dataset.py", line 541, in __init__
variables, coord_names, dims, indexes = merge_data_and_coords(
File "/home/harry/code/xarray/xarray/core/merge.py", line 466, in merge_data_and_coords
return merge_core(
File "/home/harry/code/xarray/xarray/core/merge.py", line 556, in merge_core
assert_unique_multiindex_level_names(variables)
File "/home/harry/code/xarray/xarray/core/variable.py", line 2363, in assert_unique_multiindex_level_names
raise ValueError("conflicting MultiIndex level name(s):\n%s" % conflict_str)
ValueError: conflicting MultiIndex level name(s):
'lat' (location), 'lat' (<this-array>)
'lon' (location), 'lon' (<this-array>)
Expected Output
The output should be the same as first concatenating the DataArrays, then extracting the dimension location:
>>> xr.concat([da, da2], dim="location")["location"]
<xarray.DataArray 'location' (location: 4)>
array([(10, 20), (11, 21), (12, 22), (13, 23)], dtype=object)
Coordinates:
* location (location) MultiIndex
- lat (location) int64 10 11 12 13
- lon (location) int64 20 21 22 23
Problem Description
>>> # da["location"] looks like a normal DataArray
>>> location = da["location"]
>>> location
<xarray.DataArray 'location' (location: 2)>
array([(10, 20), (11, 21)], dtype=object)
Coordinates:
* location (location) MultiIndex
- lat (location) int64 10 11
- lon (location) int64 20 21
>>> # but in actual fact, the variable._data is a MultiIndex
>>> location.variable._data
PandasIndexAdapter(array=MultiIndex([(10, 20),
(11, 21)],
names=['lat', 'lon']), dtype=dtype('O'))
This is why an error is thrown: variable.assert_unique_multiindex_level_names gets passed two variables: location.variable (the DataArray data values), and also location["location"].variable (the coordinate values), which are both MultiIndexes.
Output of xr.show_versions()
xarray: 0.14.1+36.gb3d3b44
pandas: 0.25.3
numpy: 1.18.0
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.9.1
distributed: 2.9.1
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
setuptools: 42.0.2.post20191201
pip: 19.3.1
conda: None
pytest: 5.3.2
IPython: None
sphinx: None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the MCVE with the concatenation path in xarray/core/concat.py, then inspect the merge and MultiIndex checks in xarray/core/merge.py and xarray/core/variable.py. Add or update a regression test for concatenating the extracted location DataArrays; done means it produces the four expected MultiIndex values without the conflicting-level-name error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100