python / python/mypy

Decorator confused by keyword-only parameter with Union, erroneously expects `<nothing>`, but only in some cases?

Open
#12,103 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

I am trying to write a decorator function that takes optional keyword arguments. I.e., it can be called like …

@decr
def func(…):
  …

… or …

@decr(param=val)
def func(…):
  …

I have gotten myself into a pickle with a variation of the second.

# test_case.py
from __future__ import annotations
from functools import wraps
from typing import Callable, Optional, Protocol, TypeVar, Union, overload

_ReturnsIntT = Callable[..., int]
_SubjectT = TypeVar("_SubjectT", bound=_ReturnsIntT)

class _DecoratedT(Protocol):
    def __call__(self, *args: int, join_str="", **kw: int) -> str:
        ...

@overload
def strintify(
    *,
    addtl: Union[int, _SubjectT] = 0,
) -> Callable[[_SubjectT], _DecoratedT]:
    ...

@overload
def strintify(
    f: _SubjectT,
) -> _DecoratedT:
    ...

def strintify(
    f: Optional[_SubjectT] = None,
    *,
    addtl: Union[int, _SubjectT] = 0,
) -> Union[Callable[[_SubjectT], _DecoratedT], _DecoratedT]:
    assert callable(f) or f is None

    def _decorator(f):
        @wraps(f)
        def _f(*args: int, join_str="", **kw: int) -> str:
            res = [f(*args, **kw)]
            if isinstance(addtl, int):
                res[0] += addtl
            else:
                res.append(addtl(*args, **kw))
            return join_str.join(str(i) for i in res)

        return _f
    return _decorator(f) if callable(f) else _decorator

@strintify  # <-- this is fine
def func1(a: int, b: int, c: int) -> int:
    return a**3 + b**2 + c

print(func1(3, 2, 1))  # prints 32
print(func1(3, 2, 1, join_str="|"))  # prints 32

@strintify(addtl=1)  # <-- #@#@#@#@# ERRORS HERE #@#@#@#@#
def func2(a: int, b: int, c: int) -> int:
    return a**3 + b**2 + c

print(func2(3, 2, 1))  # prints 33
print(func2(3, 2, 1, join_str="|"))  # prints 33

@strintify(addtl=lambda a, b, c: c**3 + b**2 + a)  # <-- this is fine
def func3(a: int, b: int, c: int) -> int:
    return a**3 + b**2 + c

print(func3(3, 2, 1))  # prints 328
print(func3(3, 2, 1, join_str="|"))  # prints 32|8

Type-checking that gives:

% mypy --config=/dev/null test_case.py
/dev/null: No [mypy] section in config file
test_case.py:53: error: Argument 1 has incompatible type "Callable[[int, int, int], int]"; expected <nothing>
Found 1 error in 1 file (checked 1 source file)
% mypy --version
mypy 0.931
% python --version
Python 3.9.10

Apologies for the long use case, I haven't been able to figure out how to reduce it yet.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the self-contained test_case.py example and run mypy 0.931 using the shown command to reproduce the error on the @strintify(addtl=1) case. Trace how the keyword-only Union overload is analyzed, then add a regression test showing that func2 is accepted while the existing func1 and func3 cases remain valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.