pydata / pydata/xarray

What should happen in these Edge Cases in reading from Zarr

Open
#10,806 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-backends topic-zarr
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What is your issue?

I was playing around with manually creating zarr stores and feeding them into xarray. I noticed some weird edge cases when a coord variable and a dimension name are not the same. I'm not sure that either of these cases are necessarily bugs, but the resulting behavior doesn't "feel right".

cc @TomNicholas

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "xarray[complete]@git+https://github.com/pydata/xarray.git@main",
#   "zarr",
#   "numpy",
# ]
# ///
#

import xarray as xr
import zarr
import numpy as np

fname = "mismatch_name_1.zarr"
z = zarr.open(fname)
z.create_array(
    "blah",
    data=np.arange(10, dtype=int),
    dimension_names=["coord_1"],
    overwrite=True,
)
z.create_array(
    "data", data=np.arange(100, step=10), dimension_names=["coord_1"], overwrite=True
)
ds = xr.open_zarr(fname, consolidated=False)
print(ds)

# <xarray.Dataset> Size: 160B
# Dimensions:  (coord_1: 10)
# Dimensions without coordinates: coord_1
# Data variables:
#     blah     (coord_1) int64 80B ...
#     data     (coord_1) int64 80B ...



#############################


fname = "mismatch_name_2.zarr"
z = zarr.open(fname)
z.create_array(
    "coord_1",
    data=np.arange(10, dtype=int),
    dimension_names=["blah"],
    overwrite=True,
)
z.create_array(
    "data", data=np.arange(100, step=10), dimension_names=["coord_1"], overwrite=True
)
ds = xr.load_dataset(fname, consolidated=False)
print(ds)

# <xarray.Dataset> Size: 160B
# Dimensions:  (coord_1: 10, blah: 10)
# Coordinates:
#     coord_1  (blah) int64 80B 0 1 2 3 4 5 6 7 8 9
# Dimensions without coordinates: blah
# Data variables:
#     data     (coord_1) int64 80B 0 10 20 30 40 50 60 70 80 90


# doesn't fail but gives wrong selection
print(ds.sel(blah=4))

# <xarray.Dataset> Size: 88B
# Dimensions:  (coord_1: 10)
# Coordinates:
#     coord_1  int64 8B 4
# Data variables:
#     data     (coord_1) int64 80B 0 10 20 30 40 50 60 70 80 90

# fails with error
# KeyError: "no index found for coordinate 'coord_1'"
ds.sel(coord_1=4)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided reproducer with xarray's open_zarr and load_dataset entry points, focusing on coordinate and dimension-name mismatches. Determine the intended selection behavior for both cases, then add regression coverage for the agreed semantics; the issue is done when the behavior is consistent and the examples no longer produce misleading selections or errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.