python / python/mypy

Overlapping overloads false positive with ParamSpec shenanigans

Open
#21,171 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-overloads topic-paramspec
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Abstract
mypy incorrectly flags overloads of functions with certain signatures related to ParamSpec shenangians that do not overlap with previous overloads as being narrower than the previous overloads and thus impossible to match.

Background
There is a notable limitation of ParamSpec that makes it difficult to annotate functions operating on a function and its parameters, as well as some keyword arguments (optional or required). Currently, the best workaround is to use Any to annotate the variadic args and kwargs, but that still loses some typing information. See the below snippet from the python docs.

[Concatenating Keyword Parameters](https://peps.python.org/pep-0612/#concatenating-keyword-parameters)
In principle the idea of concatenation as a means to modify a finite number of positional parameters could be expanded to include keyword parameters.

```python
def add_n(f: Callable[P, R]) -> Callable[Concatenate[("n", int), P], R]:
  def inner(*args: P.args, n: int, **kwargs: P.kwargs) -> R:
    # use n
    return f(*args, **kwargs)
  return inner
```

However, the key distinction is that while prepending positional-only parameters to a valid callable type always yields another valid callable type, the same cannot be said for adding keyword-only parameters. As alluded to [above](https://peps.python.org/pep-0612/#above) , the issue is name collisions. The parameters `Concatenate[("n", int), P]` are only valid when P itself does not already have a parameter named `n`.

```python
def innocent_wrapper(f: Callable[P, R]) -> Callable[P, R]:
  def inner(*args: P.args, **kwargs: P.kwargs) -> R:
    added = add_n(f)
    return added(*args, n=1, **kwargs)
  return inner

@innocent_wrapper
def problem(n: int) -> None:
  pass
```

Calling `problem(2)` works fine, but calling `problem(n=2)` leads to a `TypeError: problem() got multiple values for argument 'n'` from the call to added inside of innocent_wrapper.

This kind of situation could be avoided, and this kind of decorator could be typed if we could reify the constraint that a set of parameters not contain a certain name, with something like:

```python
P_without_n = ParamSpec("P_without_n", banned_names=["n"])

def add_n(
  f: Callable[P_without_n, R]
) -> Callable[Concatenate[("n", int), P_without_n], R]: ...
```

The call to add_n inside of innocent_wrapper could then be rejected since the callable was not guaranteed not to already have a parameter named n.

However, enforcing these constraints would require enough additional implementation work that we judged this extension to be out of scope of this PEP. Fortunately the design of ParamSpecs are such that we can return to this idea later if there is sufficient demand.

This indicates that the below construct is unfortunately unsupported:

class PriorityPool:
    async def execute[T, **P](self, func: Callable[P, T], /, *args: P.args, priority: int = 0, **kwargs: P.kwargs) -> T: ...
    # type checker error!

To Reproduce
mypy w/ default config on the following:

# test.pyi
from _collections_abc import Callable
from typing import Any, overload
@overload
def bar[T, **P](f: Callable[P, T], *a: P.args, **k: P.kwargs) -> T: ...
@overload
def bar[T](f: Callable[..., T], *a: Any, baz: int, **k: Any) -> T: ...

Expected Behavior
No issues.
The first overload is for when baz is not passed, and the variadic *a and **k will eventually passed to f, so the ParamSpec dictates that only arguments accepted by the function f should be passed. The second overload is the case where baz is passed as a keyword argument, but due to the above limitation, it is not possible to annotate *a: P.args and **k: P.kwargs, so they are annotated as Any. The overloads are mutually exclusive.

Actual Behavior
mypy emits an error.
test.pyi:6: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader [overload-cannot-match]

Your Environment

  • Mypy version used: 1.19.1; compiled
  • Command used: mypy test.pyi
  • Mypy configuration options from mypy.ini (and other config files): None
  • Output of python -vv: Python 3.14.0 (tags/v3.14.0:ebf955d, Oct 7 2025, 10:15:03) [MSC v.1944 64 bit (AMD64)]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report with the shown test.pyi and run mypy test.pyi using the default configuration. Trace the overload-cannot-match diagnostic for the second overload, using the ParamSpec and Any signatures in the example as the acceptance case; done means the command reports no issues for this file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.