pydata / pydata/xarray

Dataset constructor always coerces 1D data variables with same name as dim to coordinates

Open
#8,959 10 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

API design topic-indexing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What is your issue?

Whilst xarray's data model appears to allow 1D data variables that have the same name as their dimension, it seems to be impossible to actually create this using the Dataset constructor, as they will always be converted to coordinate variables instead.

We can create a 1D data variable with the same name as it's dimension like this:

In [9]: ds = xr.Dataset({'x': 0})

In [10]: ds
Out[10]: 
<xarray.Dataset> Size: 8B
Dimensions:  ()
Data variables:
    x        int64 8B 0

In [11]: ds.expand_dims('x')
Out[11]: 
<xarray.Dataset> Size: 8B
Dimensions:  (x: 1)
Dimensions without coordinates: x
Data variables:
    x        (x) int64 8B 0

so it seems to be a valid part of the data model.

But I can't get to that situation from the Dataset constructor. This should create the same dataset:

In [15]: ds = xr.Dataset(data_vars={'x': ('x', [0])})

In [16]: ds
Out[16]: 
<xarray.Dataset> Size: 8B
Dimensions:  (x: 1)
Coordinates:
  * x        (x) int64 8B 0
Data variables:
    *empty*

But actually it makes x a coordinate variable (and implicitly creates a pandas Index for it). This means that in this case there is no difference between using the data_vars and coords kwargs to the constructor:

ds = xr.Dataset(coords={'x': ('x', [0])})

In [18]: ds
Out[18]: 
<xarray.Dataset> Size: 8B
Dimensions:  (x: 1)
Coordinates:
  * x        (x) int64 8B 0
Data variables:
    *empty*

This all seems weird to me. I would have thought that if a 1D data variable is allowed, we shouldn't coerce to making it a coordinate variable in the constructor. If anything that's actively misleading.

Note that whilst this came up in the context of trying to avoid auto-creation of 1D indexes for coordinate variables, this issue is actually separate. (xref https://github.com/pydata/xarray/pull/8872#issuecomment-2027571714)

cc @benbovy who probably has thoughts

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the behavior with the Dataset constructor examples in the issue, then compare it with expand_dims('x'). Trace the constructor path that classifies a 1D variable named like its dimension. Done means data_vars={'x': ('x', [0])} preserves x as a data variable without implicitly creating a coordinate or index.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.