Hooks for custom attribute handling in xarray operations
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Over in #964, I am working on a rewrite/unification of the guts of xarray's logic for computation with labelled data. The goal is to get all of xarray's internal logic for working with labelled data going through a minimal set of flexible functions which we can also expose as part of the API.
Because we will finally have all (or at least nearly all) xarray operations using the same code path, I think it will also finally become feasible to open up hooks allowing extensions how xarray handles metadata.
Two obvious use cases here are units (#525) and automatic maintenance of metadata (e.g., cell_methods or history fields). Both of these are out of scope for xarray itself, mostly because the specific logic tends to be domain specific. This could also subsume options like the existing keep_attrs on many operations.
I like the idea of supporting something like NumPy's __array_wrap__ to allow third-party code to finalize xarray objects in some way before they are returned. However, it's not obvious to me what the right design is.
- Should we lookup a custom attribute on subclasses like
__array_wrap__(or__numpy_ufunc__) in NumPy, or should we have a system (e.g., unilaterally or with a context manager andxarray.set_options) for registering hooks that are then checked on all xarray objects? I am inclined toward the later, even though it's a little slower, just because it will be simpler and easier to get right - Should these methods be able to control the full result objects, or only set
attrsand/orname? - To be useful, do we need to allow extensions to take control of the full operation, to support things like automatic unit conversion? This would suggest something closing to
__numpy_ufunc__, which is a little more ambitious than what I had in mind here.
Feedback would be greatly appreciated.
CC @darothen @rabernat @jhamman @pwolfram
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing #964 and the existing keep_attrs and xarray.set_options entry points; the issue names no files or tests. Compare the proposed hook models with NumPy's array_wrap and numpy_ufunc, and use units, cell_methods, and history as the stated use cases. Done requires an agreed hook scope and API design.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- backend-api-design, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100