python / python/mypy

Endless call cycle between is_callable_compatible and is_protocol_implementation

Open
#21,739 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug crash topic-protocols topic-recursive-types
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Discovered by @glyph and I see it's not his first rodeo (https://github.com/python/mypy/issues/18195 for instance).

My best interpretation (which could be wrong) is that a generic class which references itself in a method's type annotation, when typed using a protocol, will cause mypy to loop endlessly between matching the protocol and checking the types in the class. I haven't pinpointed the exact mechanism but it kind of makes sense that this could happen.

To Reproduce

The original code is here: https://github.com/glyph/OmniFocusStreamCounter/commit/454d0727363595b770cb150cc08a0018bc04a783

But this is about as far as I could reduce it:

from dataclasses import dataclass
from typing import Any, Callable, Protocol


class PExp[T](Protocol):
    def EQ(self, other: T) -> PExp[T]: ...


class PRef[T](Protocol):
    def EQ(self, other: T) -> PExp[T]: ...


@dataclass
class Exp[A, B]:
    _left: A
    _op: str
    _right: B

    def EQ(self, other: Any) -> Exp[Exp[A, B], Any]:
        return Exp(self, "EQ", other)


@dataclass
class Ref[T]:
    get: Callable[[], T]

    def EQ(self, other: T) -> Exp[Ref[T], T]:
        return Exp(self, "EQ", other)


@dataclass
class Cache:
    def spam(self, name: str) -> PRef[str]:
        return Ref(lambda: "spam")

Expected Behavior

So, if you make Exp.EQ untyped, then mypy will succeed (I fixed some problems in the original code 😉 ) This is obviously the expected behaviour.

Actual Behavior

Running mypy from HEAD, uncompiled, with --show-traceback - it hangs, then on Ctrl-C we see, this, endlessly:

  File "/home/dhd/work/mypy/mypy/subtypes.py", line 1281, in is_protocol_implementation                                                              07:44:34 [233/1846]
    is_compat = is_subtype(                                                                                                                                             
        subtype, supertype, ignore_pos_arg_names=ignore_names, options=options                                                                                          
    )                                                                                                                                                                   
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 189, in is_subtype                                                                                                  
    return _is_subtype(left, right, subtype_context, proper_subtype=False)                                                                                              
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 376, in _is_subtype                                                                                                 
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dhd/work/mypy/mypy/types.py", line 2361, in accept
    return visitor.visit_callable_type(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 740, in visit_callable_type
    return is_callable_compatible(
        left,
    ...<8 lines>...
        ),
    )
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 1785, in is_callable_compatible
    if not ignore_return and not is_compat_return(left.ret_type, right.ret_type):
                                 ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 448, in _is_subtype
    return is_subtype(left, right, subtype_context=self.subtype_context)
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 189, in is_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=False)
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 376, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dhd/work/mypy/mypy/types.py", line 1684, in accept
    return visitor.visit_instance(self)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/dhd/work/mypy/mypy/subtypes.py", line 627, in visit_instance
    if right.type.is_protocol and is_protocol_implementation(
                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        left, right, proper_subtype=self.proper_subtype, options=self.options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ):
    ^

Your Environment

  • Mypy version used: mypy 2.4.0+dev.67eecf07fea3fa4bbaab9d46cd28372a8ded330b (compiled: no)
  • Mypy command-line flags: --show-traceback -v
  • Mypy configuration options from mypy.ini (and other config files):
[tool.mypy]
namespace_packages = true
plugins = ["mypy_zope:plugin"]

[[tool.mypy.overrides]]
ignore_missing_imports = true
module = [
  "objc",
  "Foundation",
  "AppKit",
  "ServiceManagement",
  "setuptools",
  "appscript",
]
  • Python version used: 3.14

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reduced Python reproducer and the traceback in mypy/subtypes.py, focusing on is_callable_compatible, is_protocol_implementation, and their calls through types.py. Reproduce the hang with --show-traceback -v, then trace the recursive subtype checks. Done means the example terminates with the expected result and regression coverage prevents the endless call cycle.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.