Duck array compatibility meeting
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Proposal: hold a high-level inter-library meeting to sort out roadblocks in the duck-array wrapping efforts.
Whilst trying to get dask, pint and xarray all working nicely together, I couldn't help but notice there are important issues which conclude with a shared sentiment that "we just need to make a decision as to what wraps what" but since then have had essentially no codified consensus, and hence no progress for the past year. Multiply-nested duck-array wrapping is complicated and involves a lot of separate libraries (as this graph of potential wrappings shows), but could be an amazingly powerful feature!
I suggest that as asynchronous discussion hasn't moved this forward, we should instead hold a (hopefully one-off) meeting to make these high-level design decisions.
I'm happy to arrange the meeting, but for this to work we ideally need attendees who understand the issues from the perspective of each of the main libraries involved - some suggestions:
- xarray (@shoyer and @keewis)
- dask (@mrocklin?)
- pint (@jthielen)
- cupy? (@jacobtomlinson?)
- sparse? (@crusaderky?)
- pytorch?? (@rgommers??)
Possible Agenda (please suggest additions!):
- Which libraries should wrap which other libraries
- Repo/NEP/etc. for standardizing wrapping order and other future decisions
- Outstanding issues to tackle first
Background reading
- Basic idea of the numpy dispatch mechanism explained in a blog post
- @jthielen 's excellent overview comment, with links to relevant NEP's
- Pint's technical commentary on array type support
Some related issues (there are many more - please add)
- https://github.com/pydata/xarray/issues/5559
- https://github.com/pydata/xarray/issues/3950
- dask/dask#5329
- dask/dask#6637
- dask/dask#6636
- dask/dask#6635
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or code entry points are named. Start with the linked Dask, xarray, Pint, and NumPy discussions and the listed background reading; done would be an agreed meeting outcome covering wrapping order, a standardization venue, and initial issues to address.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100