python / python/mypy

overloaded __new__ ignored when introducing __init__

Open
#17,251 2 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report
I'm hitting a problem where the return type of an overloaded __new__ is ignored once an __init__ is added. I've searched the issue tracker thoroughly and while I've found a ton of issues related to __new__ return types I haven't found one that seems to be the same thing.

To Reproduce

from __future__ import annotations

from typing import (
    Any,
    Generic,
    TypeVar,
    overload,
)

BE = TypeVar("BE", bound=BaseException)
E = TypeVar("E", bound=Exception)


class MyBaseClass(Generic[BE]):
    @overload
    def __new__(cls, arg: type[E]) -> MyClass[E]:
        ...
    @overload
    def __new__(cls, arg: type[BE]) -> MyBaseClass[BE]:
        ...

    def __new__(cls, arg: Any) -> Any:
        if issubclass(arg, Exception):
            return MyClass(arg)
        return MyBaseClass(arg)

    def __init__(self, arg: type[BE]):  # pyright: ignore[reportInconsistentConstructor]
        self.arg = arg

class MyClass(MyBaseClass[E]):
    def __init__(self, arg: type[E]):  # pyright: ignore[reportInconsistentConstructor]
        self.arg = arg


reveal_type(MyBaseClass(ValueError))
reveal_type(MyBaseClass(KeyboardInterrupt))

I've also found it ~impossible to type the signatures for the __new__ and __init__ implementation, but that's a separate issue.

Actual Behavior

$  mypy foo.py                                             
foo.py:36: note: Revealed type is "foo.MyBaseClass[builtins.ValueError]"
foo.py:37: note: Revealed type is "foo.MyBaseClass[builtins.KeyboardInterrupt]"

pyright handles it without issue.

$ pyright foo.py
  ./foo.py:36:13 - information: Type of "MyBaseClass(ValueError)" is "MyClass[ValueError]"
  ./foo.py:37:13 - information: Type of "MyBaseClass(KeyboardInterrupt)" is "MyBaseClass[KeyboardInterrupt]" 

and if I remove the __init__ from MyBaseClass then mypy handles the overloads correctly:

$  mypy foo.py
foo.py:36: note: Revealed type is "foo.MyClass[builtins.ValueError]"
foo.py:37: note: Revealed type is "foo.MyBaseClass[builtins.KeyboardInterrupt]"
Success: no issues found in 1 source file

Your Environment

  • Mypy version used: 1.10.0
  • Mypy command-line flags: none
  • Mypy configuration options from mypy.ini (and other config files): none
  • Python version used: 3.11.8

The source of the problem is trying to improve the typing of trio.testing.RaisesGroup: https://github.com/python-trio/trio/pull/2989

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the reproduction in foo.py with mypy 1.10.0 and compare the revealed types with the expected output and pyright's results. Trace how mypy handles overloaded new when init is present; done means the two reveal_type results match the expected MyClass[ValueError] and MyBaseClass[KeyboardInterrupt] types without regressing the case where init is absent.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.