xarray-contrib / xarray-contrib/flox
Using Flox with cubed
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 137
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Description
What would need to happen to use flox with cubed instead of dask?
(I see this question as part of fully solving https://github.com/pydata/xarray/issues/6807.)
In the code I see blockwise being used, but also dask.array.reductions._tree_reduce and custom HighLevelGraph objects being created. Is there any combination of arguments to flox that only uses blockwise? Could more of flox be made to work via the blockwise abstraction? Should we add ._tree_reduce to our list of useful computation patterns for chunked arrays that includes blockwise, map_blocks, and apply_gufunc?
Sorry if we already discussed this elsewhere.
EDIT: cc @tomwhite
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files are named. Start by tracing flox's uses of blockwise, dask.array.reductions._tree_reduce, and custom HighLevelGraph objects, then compare those computation paths with cubed's supported abstractions. Done should be a clear determination of whether flox can use blockwise alone, what changes are required otherwise, and whether _tree_reduce should be treated as a supported pattern.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100