open_mfzarr files + intake-xarray
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
This is related to a previous issue (#3668), although the actual problem on that issue is a bit more technically involved and is related with clusters... I decided to open this related issue so that the discussion in #3668 remains visible for other users.
This issue is about the need to implement code that allows to read multiple zarr files. This can be particularly helpful when reading data through an Intake Catalog entry (Intake-xarray plugin), which can allow for a compact way to introduce parallelism when working with multiple zarr files. There are two steps for this, one to work at the xarray level (write a fuction that does that) and then write an option on the Intake-xarray plugin that can use such xarray functionality.
I am more than willing to work on such problem, if nobody's already working on this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the xarray-level functionality for reading multiple zarr files, then inspect the Intake-xarray plugin integration described in the issue. Confirm how the two layers should support parallel access through an Intake Catalog, with completion requiring both the xarray function and the plugin option.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100