python / python/mypy

Poor interactions with staticmethod and overload/union

Open
#15,015 9 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-descriptors
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Behavior changed in v1.2.0 regarding how staticmethods are handled when invoked by doing foo = staticmethod(bar) (i.e. not as the @decorator syntax)

A realworld example of this appeared in matplotlib, which is wrapping a third party method as a staticmethod of one of our classes (pyparsing)

Possibly related:

#14953 (most recent change to staticmethod handling)
#14902 (typeshed sync, mentions staticmethod
#7781 (original report of staticmethod behavior

A pure python example (and extension) is provided below.

To Reproduce


from typing import overload, Callable, Any
@overload
def foo(a: int) -> None:
    ...
    
@overload
def foo(a: str) -> None: ...

def foo(a) -> None:
    ...

def func_factory(spam) -> Callable[[], Any]| Callable[[int], Any]:
    return lambda: None
    
def qux(i: int) -> int:
    return i

class Foo():
    foo = staticmethod(foo)
    bar = staticmethod(func_factory(None))
    
    @overload
    @staticmethod
    def baz(a: str) -> str: ...
    @overload
    @staticmethod
    def baz(a: int) -> int: ...
    
    @staticmethod
    def baz(a):
        return a
        
    qux = staticmethod(qux)

reveal_type(foo)  # Correctly 'Overload(def (a: builtins.int), def (a: builtins.str))'
reveal_type(Foo.foo) # Should match above, actually 'def (*Any, **Any) -> Any' on v1.1.1 and "def (a: builtins.int)" on v1.2.0

reveal_type(func_factory(None)) # Correctly 'Union[def () -> Any, def (builtins.int) -> Any]'
reveal_type(Foo.bar) # Should match above, actually 'def (*Any, **Any) -> Any' on v1.1.1 and "def (*<nothing>, **<nothing>) -> Any" on v1.2.0

reveal_type(Foo.baz) # Correctly "Overload(def (a: builtins.str) -> builtins.str, def (a: builtins.int) -> builtins.int)"

reveal_type(qux)  # Correctly "def (i: builtins.int) -> builtins.int"
reveal_type(Foo.qux) # Should match above, actually 'def (*Any, **Any) -> builtins.int' on v1.1.1 and correct on v1.2.0

Foo.foo("a") # Errors on v1.2.0

https://mypy-play.net/?mypy=latest&python=3.10&gist=c50c0fb3a6c10fbea317a31453b40e29

Expected Behavior

I would expect that the overload (or union) would carry forward to the decorated static method.

Actual Behavior

Prior to v1.2.0 (tested explicitly with v1.1.1:
All staticmethods constructed without @decorator syntax would simply get def (*Any, **Any) -> Any, which was at least not an error, though was throwing away type information.

In the case of a single non-overloaded function defined outside of the class, it gets the return type but not the parameter type.

In v1.2.0:

In the case of an overloaded method defined outside of the class, only the first overloaded definition is kept, resulting in incorrect errors for alternate signatures.

In the case of a Union type (such as from a factory function), an error is thrown on the call to staticmethod, even if all branches of the union satisfy the type staticmethod expects.
This was the actual case I saw with matplotlib.

In the case of a single non-overloaded function or an overload defined in the class (with @staticmethod on all overload branches) v1.2.0 behaves as expected.

main.py:21: error: Argument 1 to "staticmethod" has incompatible type "Union[Callable[[], Any], Callable[[int], Any]]"; expected "Callable[[VarArg(<nothing>), KwArg(<nothing>)], Any]"  [arg-type]
main.py:36: note: Revealed type is "Overload(def (a: builtins.int), def (a: builtins.str))"
main.py:37: note: Revealed type is "def (a: builtins.int)"
main.py:39: note: Revealed type is "Union[def () -> Any, def (builtins.int) -> Any]"
main.py:40: note: Revealed type is "def (*<nothing>, **<nothing>) -> Any"
main.py:42: note: Revealed type is "Overload(def (a: builtins.str) -> builtins.str, def (a: builtins.int) -> builtins.int)"
main.py:44: note: Revealed type is "def (i: builtins.int) -> builtins.int"
main.py:45: note: Revealed type is "def (i: builtins.int) -> builtins.int"
main.py:47: error: Argument 1 has incompatible type "str"; expected "int"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

  • Mypy version used: v1.2.0 (compared to v1.1.1)
  • Mypy command-line flags: N/A
  • Mypy configuration options from mypy.ini (and other config files):N/A
  • Python version used: 3.11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pure Python reproduction in the issue and compare mypy 1.1.1, 1.2.0, and the linked mypy-play example. Trace staticmethod handling for externally defined overloads and union-valued factories, then add focused regression coverage showing that overloads and compatible union branches retain the expected callable types without rejecting Foo.foo("a").

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.