Storing Sparse arrays to Zarr
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 668
- Forks
- 141
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Periodically we have had users request some way to store sparse arrays with Zarr. TBH this is actually pretty doable generally today as demonstrated by this comment. Also this strategy works nicely with Sparse. Admittedly these examples are showing how this works with in-memory Zarr Arrays. Though this would work just as well with any MutableMapping derived store.
Given there seems to be a fair bit of interest in being able to work with more general N-D sparse arrays and having a flexible way to store them, am wondering if it makes sense to provide some functionality in Sparse to store data in Zarr Arrays. Happy to answer any questions. Also would be interested to hear thoughts on this proposal. :)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked Zarr examples and the referenced Sparse discussion to understand the proposed storage approach. The issue does not name files, tests, or an entry point. Done would mean agreeing on and implementing a supported way for Sparse arrays to be stored in Zarr-compatible stores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100