pydata / pydata/xarray

NumPy `__array_ufunc__` does not work with typing

Open
#6,524 7 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic-typing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What is your issue?

When using NumPy functions which have been patched to work with xarray objects using __array_ufunc__, typing tools like MyPy do not correctly calculate the return value.

For example, the function np.exp has been adapted by xarray to return a DataArray if it is passed a DataArray, so that code like the following will work:

import xarray as xr
import numpy as np

da = xr.DataArray([1, 2, 3, 4])
blah = np.exp(da).rename("blah")

This code creates an xr.DataArray, uses numpy to calculate the exp of its values and then, since it is returned as a xr.DataArray it can call xarray methods like rename on it.

However, running MyPy on this code gives the error:

error: "ndarray[Any, dtype[Any]]" has no attribute "rename"

This is because there is typing information from NumPy which claims that np.exp returns ndarray[Any, dtype[Any]].

Now, I'm unsure whether this is a bug in xarray not providing the typing information to the __array_ufunc__ code, a bug in NumPy for not having the flexibility to type these overrides or in MyPy for now allowing for overrides like this to happen but I wanted to star with the place where the observed error occurs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the MyPy error with the np.exp(DataArray).rename("blah") example, then inspect xarray's array_ufunc behavior alongside NumPy's np.exp typing. Done means the return type is inferred as an xarray DataArray and the rename call type-checks, with a regression test covering the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
developer-experience
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.