Updating DataArray attributes in a Zarr archive does not work as for Dataset
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
When using the new functionality of calling to_zarr on DataArrays, I've noticed that the attributes are not updated and can only be set once, on initial archive creation. In our case, we are performing sequential writes to the Zarr regions and also want to update attributes. Regular Zarr API allows updating metadata and it also works with Datasets, so we suspect it's an issue with DataArrays. In fact, when calling to_zarr, the DataArray is converted to a Dataset, but when we check it, it does not have its attributes, as they live inside the nested array.
What did you expect to happen?
I would like to update attributes in the same manner as with plain Zarr or with Datasets. Currently, we have to resort to modifying the .zmetadata JSON and consider this dangerous and hacky.
Minimal Complete Verifiable Example
import xarray as xr
ds_path = "ds"
xr.Dataset().assign_attrs(a=1).to_zarr(ds_path, mode="w")
xr.open_dataset(ds_path, engine="zarr").assign_attrs(b=2).to_zarr(ds_path, mode="a")
da_path = "da"
xr.DataArray().assign_attrs(a=1).to_zarr(da_path, mode="w")
xr.open_dataarray(da_path, engine="zarr").assign_attrs(b=2).to_zarr(da_path, mode="a")
print(xr.open_dataset(ds_path, engine="zarr").attrs == xr.open_dataarray(da_path, engine="zarr").attrs)
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
No response
Anything else we need to know?
No response
Environment
INSTALLED VERSIONS
commit: None
python: 3.11.3 (tags/v3.11.3:f3909b8, Apr 4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 183 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: ('Polish_Poland', '1250')
libhdf5: None
libnetcdf: None
xarray: 2023.7.0
pandas: 2.0.3
numpy: 1.25.2
scipy: 1.11.2
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.16.1
cftime: None
nc_time_axis: None
PseudoNetCDF: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.7.2
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 68.0.0
pip: 23.2.1
conda: None
pytest: None
mypy: None
IPython: 8.14.0
sphinx: 7.1.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the minimal example using DataArray.to_zarr and compare its behavior with Dataset.to_zarr. Trace the DataArray-to-Dataset conversion and Zarr metadata update path; the issue is done when sequential DataArray writes update attributes like Dataset writes do, with a regression test covering the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100