pydata / pydata/xarray

No way to specify compat behaviour for xr.apply_ufunc and ops that use it; binary arithmetic ufuncs don't use arithmetic_compat.

Open
#11,272 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Is your feature request related to a problem?

We're now able to override the problematic (see https://github.com/pydata/xarray/issues/9481, https://github.com/pydata/xarray/issues/10924) compat='minimal' default for arithmetic using e.g. xr.set_options(arithmetic_compat='override'):

import xarray as xr
a = xr.DataArray(1, coords={'foo': 1})
b = xr.DataArray(1, coords={'foo': 2})

with xr.set_options(arithmetic_compat='override'):
     print(a + b)

<xarray.DataArray ()> Size: 8B
array(2)
Coordinates:
    foo      int64 8B 1

Unfortunately there's no way to set compat for xr.apply_ufunc, or for any of the multi-argument ufuncs based on it which perform compat checks

with xr.set_options(arithmetic_compat='override'):
     print(xr.apply_ufunc(lambda x, y: x+y, a, b))

<xarray.DataArray ()> Size: 8B
array(2)

with xr.set_options(arithmetic_compat='override'):
     print(xr.where(True, a, b))

<xarray.DataArray ()> Size: 8B
array(1)

with xr.set_options(arithmetic_compat='override'):
     print(xr.ufuncs.add(a, b))

<xarray.DataArray ()> Size: 8B
array(1)

This is particularly problematic because the behavior now differs for arithmetic operators (going via _binary_op) and arithmetic ufuncs like xr.ufuncs.add or np.add, which go via apply_ufunc.

Describe the solution you'd like / alternatives you've considered

Any views on these options:

(A) : Add an explicit compat argument to apply_ufunc, defaulting to the current behaviour of 'minimal', and let multi-argument ufuncs pass this argument on. (xr.ufuncs based on _binary_ufunc already pass through extra kwargs, but some like xr.dot, xr.where, da.where, da.fillna would need updating.)

(B): Make it configurable via xr.set_options. It could share the existing arithmetic_compat setting which defaults to minimal, or add a new ufunc_compat setting.

(C): (A)+(B): there's an explicit argument for it but it defaults to e.g. OPTIONS["ufunc_compat"]

In all cases I'd suggest setting the defaults to change in a future version to 'override', along with the other planned compat default changes.

If we used the arithmetic_compat setting for apply_ufunc, it'd be reasonable to ask why not for consistency also use arithmetic_join as the default for apply_ufunc's join. However its default is currently different (exact for apply_ufunc, inner for arithmetic), so maybe using a separate ufunc_compat side-steps this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with xr.apply_ufunc and the multi-argument ufunc paths described in the issue, including _binary_ufunc, xr.dot, xr.where, da.where, and da.fillna. Compare how compat is currently selected with arithmetic operators and identify the affected tests or callers. Done means the chosen explicit or option-based compat behavior is applied consistently without changing unrelated defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
api, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.