python / python/mypy

Faulty Protocol-TypeVar interactions

Open
#9,436 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

🐛 Bug Report

A bit of background: I'm trying the to describe certain operations (e.g. addition) of the numpy.ndarray class.
As the annotations of most operations are highly dependent on the type of underlying scalar, it is easier to let
aforementioned scalars handle all the details and dispatch results the actual ndarray class using protocols.

However, when trying to implementing this I encountered some odd interaction between the, herein created,
__radd__-protocol and a typevar in the scalars' actual __radd__ method:

from typing import Protocol, TypeVar, Generic

# Define a protocol for `__radd__`
T_co = TypeVar("T_co", covariant=True)
T_contra = TypeVar("T_contra", contravariant=True)

class SupportsRAdd(Protocol[T_contra, T_co]):
    def __radd__(self, other: T_contra) -> T_co: ...

With the protocol in hand we can define our scalar (e.g. bool_) and the array used for embedding the scalar:

class generic: ...  # baseclass for numpy scalars

T1 = TypeVar("T1", bound=generic)
T2 = TypeVar("T2", bound=generic)

class bool_(generic):
    def __radd__(self, other: T1) -> T1: ...
    def __add__(self, other: T1) -> T1: ...

class ndarray(Generic[T1]):
    def __add__(self, other: SupportsRAdd[T1, T2]) -> ndarray[T2]: ...

So far so good.
Additions involving ndarray[bool_] should now now be described by the underlying bool_, which is more or less what happens:


array: ndarray[bool_]
scalar: bool_

# note: Revealed type is 'op.bool_*'
reveal_type(scalar + scalar)

# error: Value of type variable "T1" of "__radd__" of "bool_" cannot be "ndarray[bool_]"  [type-var]
# note: Revealed type is 'op.ndarray*[op.bool_]'
reveal_type(array + scalar)

In both cases the return-type is correctly inferred, but in the array + scalar example it is accompanied by an error (a false positive?). Note removing the typevars in bool_.__add__ & .__radd__ does get rid of the error, but that's not exactly a viable solution due the sheer amount of generic subclasses.

Expected Behavior

Reveal the return type without any errors.

Actual Behavior

The return type is revealed in addition to producing a [type-var]-related error.

Your Environment

  • Mypy version used: mypy 0.782
  • Mypy command-line flags: --show-error-codes (including or excluding --strict makes no difference here)
  • Mypy configuration options from mypy.ini (and other config files): n.a.
  • Python version used: Python 3.8.3
  • Operating system and version: macOS 10.15.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Protocol, TypeVar, Generic, ndarray, and bool_ reproducer with mypy and --show-error-codes. Investigate the add and radd interaction that emits the type-var error; done means the revealed return type remains correct without producing that error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.