stack().unstack() not the same as original for datavars dependent on single coordinate of multi_index
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What is your issue?
(See MVCE example)
The combination ds.stack().unstack() doesn't entirely give back the original ds, when there's a datavariable that only depends on a subset of coords of the multi-index used for stacking.
- Is this on purpose? And if so, what's the rationale?
- I would imagine that it could also be more memory efficient, when the original indexes
xandyare kept that make up the multi-index(midx=[x,y])after astack()operation. Because then you don't have to express and thus repeat the values of dataarrays that only depend on a subset of the indexes that make up the multi-index.
MVCE
# xarray==2022.11.0
import xarray as xr
ds = xr.Dataset(coords={'x':[1,2], 'y':[3,4]})
ds['a'] = ds.x + 5
# <xarray.Dataset>
# Dimensions: (x: 2, y: 2)
# Coordinates:
# * x (x) int32 1 2
# * y (y) int32 3 4
# Data variables:
# a (x) int32 6 7
ds_stacked = ds.stack(midx=['x','y'])
# <xarray.Dataset>
# Dimensions: (midx: 4)
# Coordinates:
# * midx (midx) object MultiIndex
# * x (midx) int32 1 1 2 2
# * y (midx) int32 3 4 3 4
# Data variables:
# a (midx) int32 6 6 7 7
ds_unstacked = ds_stacked.unstack()
# <xarray.Dataset>
# Dimensions: (x: 2, y: 2)
# Coordinates:
# * x (x) int32 1 2
# * y (y) int32 3 4
# Data variables:
# a (x, y) int32 6 6 7 7
Expected
ds_unstacked to be the same as ds.
Instead the variable a has now also become a function of coordinate y, but that's not entirely correct.
I.e., after ds.stack(), that the variable 'a' is still only dependent on the original coordinate 'x', which is just a part of the multi-index.
ds_stacked = ds.stack(midx=['x','y'])
# <xarray.Dataset>
# Dimensions: (midx: 4)
# Coordinates:
# * midx (midx) object MultiIndex
# * x (midx) int32 1 1 2 2
# * y (midx) int32 3 4 3 4
# Data variables:
# a (x) int32 6 6 7 7
Maybe for clarity
# <xarray.Dataset>
# Dimensions: (midx: 4)
# Coordinates:
# * midx (midx) object MultiIndex
# * x (midx) int32 1 1 2 2
# * y (midx) int32 3 4 3 4
# Data variables:
# a (midx.x) int32 6 6 7 7
Or maybe to save memory
Make a relation/difference between midx.x (repeated values of x due to stacking) and x (original unique values).
# <xarray.Dataset>
# Dimensions: (midx: 4)
# Coordinates:
# * midx (midx) object MultiIndex
# * x (midx) int32 1 1 2 2
# * y (midx) int32 3 4 3 4
# Data variables:
# a (x) int32 6 7
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the MVCE and inspect the behavior of Dataset.stack() and Dataset.unstack() for variables that depend on only one component of a MultiIndex. Determine and document the intended round-trip and memory semantics, then add regression coverage showing the chosen behavior for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100