API Design for Xarray Backends
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
It has come time to formalize the API for Xarray backends. We now have the following backends implemented in xarray:
| Backend | Read | Write |
|---|---|---|
| netcdf4-python | x | x |
| h5netcdf | x | x |
| pydap | x | |
| pynio | x | |
| scipy | x | x |
| rasterio* | x | |
| zarr | x | x |
* currently does not inherit from backends.AbstractDatastore
And there are conversations about adding additional backends, for example:
- TileDB: https://github.com/pangeo-data/storage-benchmarks/issues/6
- PseudoNetCDF: #1905
However, as anyone who has worked on implementing or optimizing any of our current backends can attest, the existing DataStore API is not particularly user/developer friendly. @shoyer asked me to open an issue to discuss what a more user friendly backend API would look like so that is what this issue will be. I have left out a thorough description of the current API because, well, I don't think it can done in a succinct manner (thats the problem).
Note that @shoyer started down a API refactor some time ago in #1087 but that effort has stalled, presumably because we don't have a well defined set of development goals here.
cc @pydata/xarray
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the current DataStore API and the stalled API refactor in issue #1087, then compare the listed read and write backends. Done means the project has agreed development goals and a formalized, more user- and developer-friendly backend API.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100