pydata / pydata/xarray

DataArray.unstack() leaving dimensions 'in order'

Open
#3,786 16 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

It's documented that when you .unstack() a DataArray, "New dimensions will be added at the end."

I'm using .unstack() kind of like np.reshape(), to split one dimension into two. From this perspective, it's surprising that new dimensions go at the end, rather than in place of the old dimension. This also means that either the new array is no longer C-contiguous, or the .unstack() operation has had to copy all the data to rearrange it. It appears I'm not the only one to think this: the code sample in #2419 (about adding a reshape method) does the same workaround that I do: unstack and then carefully reorder the dimensions.

First, I'd like to understand: why the end? It seems like there's an obvious place to put unstacked dimensions (ab, c -> a, b, c), but presumably there's some reason it doesn't work that way.

Then, is there anything one can do with xarray today to 'unstack' a dimension while keeping the order, short of using .transpose() afterwards to recreate the desired order? And if not, is there any potential to add a feature for that?

I guess this might touch on missing data: unstack() can create and handle 'gaps', whereas reshape() can't. If so, is there some more limited operation that could only unstack a perfectly regular multiindex - where it's guaranteed to separate into 2 dimensions without creating gaps?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names DataArray.unstack(), np.reshape(), transpose(), and the discussion in #2419; start by reading those APIs and the existing dimension-order behavior. A useful outcome is a decision on whether preserving the original position is feasible, especially for regular MultiIndexes and cases with gaps.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.