pydata / pydata/xarray

Align with join='override' may update index coordinate metadata

Open
#6,860 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-metadata
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened?

It seems that align(*, join="override") may have affected and still may affect the metadata of index coordinate data in an incorrect way. See the MCV example below.

cf. @keewis' original https://github.com/pydata/xarray/pull/6857#discussion_r934425142.

What did you expect to happen?

Index coordinate metadata unaffected by alignment (i.e., metadata is passed through object -> aligned object for each object), like for align with other join methods.

Minimal Complete Verifiable Example
import xarray as xr


ds1 = xr.Dataset(coords={"x": ("x", [1, 2, 3], {"foo": 1})})
ds2 = xr.Dataset(coords={"x": ("x", [1, 2, 3], {"bar": 2})})


aligned1, aligned2 = xr.align(ds1, ds2, join="override")

aligned1.x.attrs
# v2022.03.0 -> {'foo': 1}
# v2022.06.0 -> {'foo': 1, 'bar': 2}
# PR #6857 -> {'foo': 1}
# expected -> {'foo': 1}

aligned2.x.attrs
# v2022.03.0 -> {}
# v2022.06.0 -> {'foo': 1, 'bar': 2}
# PR #6857 -> {'foo': 1, 'bar': 2}
# expected -> {'bar': 2}


aligned11, aligned22 = xr.align(ds1, ds2, join="inner")

aligned11.x.attrs
# {'foo': 1}

aligned22.x.attrs
# {'bar': 2}
MVCE confirmation
  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output

No response

Anything else we need to know?

No response

Environment
INSTALLED VERSIONS ------------------ commit: None python: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:36:15) [Clang 11.1.0 ] python-bits: 64 OS: Darwin OS-release: 20.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1

xarray: 0.21.2.dev137+g30023a484
pandas: 1.4.0
numpy: 1.22.2
scipy: 1.7.1
netCDF4: 1.5.8
pydap: installed
h5netcdf: 0.11.0
h5py: 3.4.0
Nio: None
zarr: 2.6.1
cftime: 1.5.2
nc_time_axis: 1.2.0
PseudoNetCDF: installed
rasterio: 1.2.10
cfgrib: 0.9.8.5
iris: 3.0.4
bottleneck: 1.3.2
dask: 2022.01.1
distributed: 2022.01.1
matplotlib: 3.4.3
cartopy: 0.20.1
seaborn: 0.11.1
numbagg: 0.2.1
fsspec: 0.8.5
cupy: None
pint: 0.16.1
sparse: 0.13.0
flox: None
numpy_groupies: None
setuptools: 57.4.0
pip: 20.2.4
conda: None
pytest: 6.2.5
IPython: 7.27.0
sphinx: 3.3.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the minimal example with xr.align(..., join="override") and compare its coordinate attributes with the documented inner-join behavior. Trace the align implementation and existing alignment tests, then add a regression test showing that each aligned object's index coordinate metadata is preserved; done means aligned1.x keeps {'foo': 1} and aligned2.x keeps {'bar': 2}.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.