Kerchunk and Zarr V3
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 366
- Forks
- 96
- Avg merge
- 24m
- Merged PRs (30d)
- 1
Description
The Zarr V3 spec is now undergoing public review and testing. This issue raises the question of how Kerchunk should integrate with the new spec.
Key changes in the V3 spec that are relevant particularly to Kerchunk (https://github.com/zarr-developers/zarr-specs/pull/149):
- change to chunk and metadata key names
- introduction of storage transformer extensions
- likely introduction of a Sharding Storage Transformer extension: https://github.com/zarr-developers/zarr-specs/pull/152
Questions:
- Has any work been done to produce kerchunk references that align with the v3 storage key conventions?
- Could Kerchunk provide be thought of as a storage transformer in v3?
- The Sharding proposal (linked above) includes some references to putting shards in hdf5 files. Could Kerchunk extend that spec?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked Zarr V3 specification PRs 149 and 152, then review the repository for existing reference-generation entry points and tests. The issue names no files or tests; done would require a decided integration approach for V3 key conventions, storage transformers, and sharding, with corresponding implementation scope identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100