python / python/mypy

`Never` inferred in complex situation with variadic callable protocol

Open
#16,522 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-pep-646
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Mypy is inferring a Never in a bit of a complicated setup involving TypeVarTuple, __call__ protocols and a regular TypeVar. If an annotated assignment is present it is able to check that they match. But when a bare call is done, it infers Never and always produces an error. This is a simplified version of Trio's Nursery.start() async spawn method, which I'm trying to type. (See src/trio/_core/_run.py.) I simplified it a little, removing an overload and making it synchronous.

To Reproduce

from typing import Generic, TypeVar, Protocol
from typing_extensions import TypeVarTuple, Unpack

PosArgT = TypeVarTuple("PosArgT")
StatusT = TypeVar("StatusT")
StatusT_co = TypeVar("StatusT_co", covariant=True)
StatusT_contra = TypeVar("StatusT_contra", contravariant=True)

class TaskStatus(Generic[StatusT_contra]):
    def started(self, value: StatusT_contra) -> None: ...

class NurseryStartFunc(Protocol[Unpack[PosArgT], StatusT_co]):
    def __call__(
        self,
        *args: Unpack[PosArgT],
        task_status: TaskStatus[StatusT_co],
    ) -> object: ...

def nursery_start(
    async_fn: NurseryStartFunc[Unpack[PosArgT], StatusT],
    *args: Unpack[PosArgT],
) -> StatusT:  ...

def task(a: int, b: str, *, task_status: TaskStatus[set[str]]) -> None: ...

def test() -> None:
    a_task: NurseryStartFunc[int, str, set[str]] = task  # task implements the protocol
    result: set[str] = nursery_start(task, 1, "b")  # If result is annotated this works.
    result = nursery_start(task, "a", 2)  # Detects invalid call correctly
    # Without assignment or if unnanotated, infers Never as return type?
    nursery_start(task, 1, "b")

https://mypy-play.net/?mypy=latest&python=3.11&gist=31be4208e294eb6c5cbc7390f4a836f4

Expected Behavior

Ideally Mypy would be able to propagate the typevar in TaskStatus to determine the return type, and verify that appropriate types were passed for *args.

Actual Behavior

(Last two statements)
main.py:39: error: Argument 1 to "nursery_start" has incompatible type "Callable[[int, str, NamedArg(TaskStatus[set[str]], 'task_status')], None]"; expected "NurseryStartFunc[str, int, set[str]]"  [arg-type]
main.py:41: error: Argument 1 to "nursery_start" has incompatible type "Callable[[int, str, NamedArg(TaskStatus[set[str]], 'task_status')], None]"; expected "NurseryStartFunc[int, str, Never]"  [arg-type]

The first error is correct, showing that Mypy can understand the types somewhat. But in the second case it's strangely producing Never, when it should really be effectively Any - the type is unused so it doesn't matter what it is.

Your Environment

  • Mypy version used: 1.7.0, also tried master
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.8, 3.11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied Python reproduction with the linked mypy-play example and compare the annotated and bare-call cases. Start by tracing type inference for TypeVarTuple, the callable protocol, and TaskStatus in this example. Done means the valid bare call no longer reports an inferred Never, while the invalid argument order is still rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.