to_zarr mode='a-', append_dim; if dim value exists raise error
Open
Nobody has claimed this yet.
topic-zarr
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
If I have a ds with time, lat, lon and I call the same command twice:
ds.to_zarr('test.zarr', append_dim='time')
ds.to_zarr('test.zarr', append_dim='time')
Can it raise an error since all the times already exist?
Kind of like:
import numpy as np
import xarray as xr
ds = xr.tutorial.open_dataset('air_temperature')
ds.to_zarr('test_air.zarr', append_dim='time')
ds_tmp = xr.open_mfdataset('test_air.zarr', engine='zarr')
overlap = np.intersect1d(ds['time'], ds_tmp['time'])
if len(overlap) > 1:
raise ValueError(f'Found overlapping values in datasets {overlap}')
ds.to_zarr('test_air.zarr', append_dim='time')
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the Dataset.to_zarr entry point and its append_dim handling, then reproduce the issue with the provided Python example using repeated writes and overlapping time values. Done means an append with existing dimension values raises a ValueError instead of writing duplicates; the payload does not name a test file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 39/100