python / python/mypy

callables are inferred to be descriptors when they shouldn't always be, protocols are inferred to not be descriptors when maybe they should, and mypy converts between them with no error

Open
#15,189 13 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Callable implicitly has a __get__ method which makes it a bindable function
when used at class scope.

Protocols with a __call__ method, on the other hand, are not assumed to
have a __get__ method, and thus, not to be functions.

However, implicit conversion is allowed between the two, leading to confusing
behavior in higher-order applications, such as decorators which wish to apply
in similar fashion to either methods or free functions. For example, consider
a decorator which returns a __call__able Protocol, in an effort to capture
the nuances of named and default arguments (since mypy_extensions.NamedArg
et. al. are clearly
deprecated
).
It needs to manually re-specify the behavior of __get__ in order to be
treated the same way as an equivalent Callable would.

To Reproduce

Consider the errors and non-errors in this program, with particular emphasis on
the incredibly subtle distinction between be_a_typevar and be_a_callable,
which behave differently:

from __future__ import annotations

from typing import Protocol, Callable, TypeVar, ParamSpec, Any


class HelloMethod(Protocol):
    def __call__(_, self: HelloSayer) -> None:
        ...

def be_a_protocol(method: HelloMethod) -> HelloMethod:
    return method

class HelloSelfless(Protocol):
    def __call__(self) -> None:
        ...

def be_selfless(method: HelloSelfless) -> HelloSelfless:
    return method

HelloVar = TypeVar("HelloVar", bound=Callable[..., Any])

def be_a_typevar(method: HelloVar) -> HelloVar:
    return method

Params = ParamSpec("Params")
Result = TypeVar("Result")

def be_a_callable(method: Callable[Params, Result]) -> Callable[Params, Result]:
    return method

class HelloSayer:

    def hello_regular(self) -> None:
        print("hello.")

    @be_a_protocol
    def hello_method(self) -> None:
        print("hello?")

    @be_selfless
    def hello_selfless(self) -> None:
        print("hello??")

    @be_a_typevar
    @be_a_protocol
    def hello_typevar(self) -> None:
        print("hello!")

    @be_a_callable
    @be_a_protocol
    def hello_callable(self) -> None:
        print("HELLO")


h = HelloSayer()
h.hello_regular()
h.hello_selfless()
h.hello_method()
h.hello_typevar()
h.hello_callable()

Proposed Solution

I think there really needs to be a FunctionType, and it should behave more or
less like Callable does today with respect to the descriptor protocol. It
should also have all the attributes that functions have, so that
metaprogramming with __code__ and __name__ and soforth doesn't need to be
littered with type:ignores.

In strict mode, I also think that this should become an error:

class Abstract(Protocol):
    def stuff(self):
        pass

class Concrete:
    var = Abstract()

abstract = Concrete().var  # Should be: Abstract has no attribute __get__

Any concrete type can trace its implementation of __get__ up through its
hierarchy to object, but Protocol might have any implementation, or no
implementation. It can't really be treated as if we can guess what it's going
to return.

In general Callable and descriptors and all the adjacent special cases are
kind of a big mess and this keeps getting reported in different ways. See
also:

And this one is only tangentially related, but it makes it more annoying to
write the __get__ for the descriptor protocol directly to work around it:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproducer in the issue and compare the Callable, Protocol, TypeVar, and ParamSpec cases, then review the linked issues about descriptors and callable behavior. Done should include a settled rule for descriptor inference and conversion, plus regression coverage for the reported errors and non-errors; no specific files or tests are named.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.