python / python/mypy

Overload ambiguity is ignored in `self`-annotated methods

Open
#11,347 5 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-overloads
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Per the title: mypy does not check for overload ambiguity in self-annotated methods.
Mypy will instead, effectively, ignore self and then return the first matching overload,
thus circumventing the usual Any that is returned when overload ambiguity is involved.
This has lead to a number of false positives in https://github.com/numpy/numpy/issues/20099 and (in part) https://github.com/numpy/numpy/issues/20072.

The conditions for triggering this bug are as following:

  • The callable is a method (things work fine for functions).
  • The method is overloaded.
  • self must be annotated with a parametrized generic.

To Reproduce

from typing import Any, Generic, overload, TypeVar, TYPE_CHECKING
import numpy as np

_CharType = TypeVar("_CharType", np.str_, np.bytes_)


class CharArray(Generic[_CharType]):
    @overload
    def strip(self: CharArray[np.str_], chars: str = ...) -> CharArray[np.str_]: ...
    @overload
    def strip(self: CharArray[np.bytes_], chars: bytes = ...) -> CharArray[np.bytes_]: ...


ar_any: CharArray[Any]
ar_str: CharArray[np.str_]
ar_bytes: CharArray[np.bytes_]

if TYPE_CHECKING:
    # The good
    reveal_type(ar_str.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"
    reveal_type(ar_bytes.strip())  # E: Revealed type is "__main__.CharArray[numpy.bytes_]"

    # The bad
    # "__main__.CharArray[Any]" would be expected here due to overload ambiguity
    reveal_type(ar_any.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"

    # Calling the method directly from the class does do the trick
    reveal_type(CharArray.strip(ar_any))  # E: Revealed type is "test.CharArray[Any]"

Your Environment

  • Mypy version used: 0.910
  • Mypy command-line flags: n.a.
  • Mypy configuration options from mypy.ini (and other config files): n.a.
  • Python version used: 3.9.6
  • Operating system and version: macOS 11.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Python reproduction with mypy and compare the revealed types for the overloaded method calls. Trace overload resolution for methods whose self parameter is a parametrized generic; done means the ambiguous CharArray[Any].strip() call returns Any while the concrete self types retain their specific results.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
compilers, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.