callables are inferred to be descriptors when they shouldn't always be, protocols are inferred to not be descriptors when maybe they should, and mypy converts between them with no error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
Callable implicitly has a __get__ method which makes it a bindable function
when used at class scope.
Protocols with a __call__ method, on the other hand, are not assumed to
have a __get__ method, and thus, not to be functions.
However, implicit conversion is allowed between the two, leading to confusing
behavior in higher-order applications, such as decorators which wish to apply
in similar fashion to either methods or free functions. For example, consider
a decorator which returns a __call__able Protocol, in an effort to capture
the nuances of named and default arguments (since mypy_extensions.NamedArg
et. al. are clearly
deprecated).
It needs to manually re-specify the behavior of __get__ in order to be
treated the same way as an equivalent Callable would.
To Reproduce
Consider the errors and non-errors in this program, with particular emphasis on
the incredibly subtle distinction between be_a_typevar and be_a_callable,
which behave differently:
from __future__ import annotations
from typing import Protocol, Callable, TypeVar, ParamSpec, Any
class HelloMethod(Protocol):
def __call__(_, self: HelloSayer) -> None:
...
def be_a_protocol(method: HelloMethod) -> HelloMethod:
return method
class HelloSelfless(Protocol):
def __call__(self) -> None:
...
def be_selfless(method: HelloSelfless) -> HelloSelfless:
return method
HelloVar = TypeVar("HelloVar", bound=Callable[..., Any])
def be_a_typevar(method: HelloVar) -> HelloVar:
return method
Params = ParamSpec("Params")
Result = TypeVar("Result")
def be_a_callable(method: Callable[Params, Result]) -> Callable[Params, Result]:
return method
class HelloSayer:
def hello_regular(self) -> None:
print("hello.")
@be_a_protocol
def hello_method(self) -> None:
print("hello?")
@be_selfless
def hello_selfless(self) -> None:
print("hello??")
@be_a_typevar
@be_a_protocol
def hello_typevar(self) -> None:
print("hello!")
@be_a_callable
@be_a_protocol
def hello_callable(self) -> None:
print("HELLO")
h = HelloSayer()
h.hello_regular()
h.hello_selfless()
h.hello_method()
h.hello_typevar()
h.hello_callable()
Proposed Solution
I think there really needs to be a FunctionType, and it should behave more or
less like Callable does today with respect to the descriptor protocol. It
should also have all the attributes that functions have, so that
metaprogramming with __code__ and __name__ and soforth doesn't need to be
littered with type:ignores.
In strict mode, I also think that this should become an error:
class Abstract(Protocol):
def stuff(self):
pass
class Concrete:
var = Abstract()
abstract = Concrete().var # Should be: Abstract has no attribute __get__
Any concrete type can trace its implementation of __get__ up through its
hierarchy to object, but Protocol might have any implementation, or no
implementation. It can't really be treated as if we can guess what it's going
to return.
In general Callable and descriptors and all the adjacent special cases are
kind of a big mess and this keeps getting reported in different ways. See
also:
- https://github.com/python/mypy/issues/14392
- https://github.com/python/mypy/issues/14869
- https://github.com/python/mypy/issues/14123
- https://github.com/python/mypy/issues/7945
- https://github.com/python/mypy/issues/7724
- https://github.com/python/mypy/issues/6700
- https://github.com/python/mypy/issues/5530
- https://github.com/python/mypy/issues/3482
- https://github.com/python/mypy/issues/2267
And this one is only tangentially related, but it makes it more annoying to
write the __get__ for the descriptor protocol directly to work around it:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproducer in the issue and compare the Callable, Protocol, TypeVar, and ParamSpec cases, then review the linked issues about descriptors and callable behavior. Done should include a settled rule for descriptor inference and conversion, plus regression coverage for the reported errors and non-errors; no specific files or tests are named.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100