to_zarr append errors when using encoding point in wrong direction
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
When I try to append to a zarr store created from encoded data it isn't clear that I need to decode the data I want to append.
When I run the attached code, I get an error:
ValueError: failed to prevent overwriting existing key add_offset in attrs. This is probably an encoding field used by xarray to describe how a variable is serialized. To proceed, remove this key from the variable's attributes manually.
Following that error, if I just remove the encoding I can append to the zarr store, but the data isn't encoding correctly.
The fix is to xr.decode_cf(data) and then append.
It would help if the to_zarr append documentation made this clearer.
Example given here:
https://github.com/cgentemann/cloud_science/blob/master/make_zarr/zarr_append_issue.ipynb
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked zarr_append_issue.ipynb to understand the reported append workflow, then find the xarray documentation for to_zarr append behavior. Clarify that data should be decoded with xr.decode_cf(data) before appending, and confirm that the documented guidance addresses the encoding error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100