Update dataset attributes and sync to disk store
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Summary: I would like to be able to update dataset/dataarray attributes and sync the updates to disk.
Regarding the Zarr backend in Xarray, at the moment once datasets are read from disk all modifications are in memory only. This means that for the particular case of metadata stored as attributes, in order to update an attribute on disk there are two choices:
- Assign the new attribute to the dataset and write the whole dataset again to disk.
- Use Zarr directly to access the dataset attributes and perform an 'attribute sync'.
The former is prohibitive , in particular for large datasets, but in general it is not a good practice that in order to modify one attribute I need to rewrite the whole pixel data.
The second is possible, and certainly I could imagine the need to write a custom function to do this. Is that something that should be available in the xarray library though? Something like ds.sync_attributes() that updates the attributes on disk?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names Xarray's Zarr backend and a proposed ds.sync_attributes() entry point, but no file or test. Start by locating the Zarr backend handling for dataset and dataarray attributes and comparing it with Zarr's attribute-sync behavior. Done means defining and implementing a supported way to persist attribute changes without rewriting pixel data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100