Support irregular array chunks using zarr
- Dominant language
- Python
- Stars
- 13.9k
- Forks
- 2k
- PR merge metrics
- No merged PRs in 30d
Description
See https://zarr.readthedocs.io/en/stable/user-guide/examples/rectilinear_chunks/
```
import dask.array as da
import zarr
zarr.config.set({"array.rectilinear_chunks": True})
X = da.concatenate((da.ones((3, 10)), da.ones((2, 10)), da.ones((4, 10))), axis=0)
da.to_zarr(X, "test.zarr") # automatically rechunks to uniform chunks
```
Contributor guide
Research direction
Start at the da.to_zarr entry point and compare the example with the Zarr rectilinear_chunks documentation linked in the issue. Trace where the concatenated array is rechunked before writing. Done means arrays with irregular chunks can be written without being automatically rechunked to uniform chunks, with coverage for the shown case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100