Adding __init__ breaks type inference from __new__
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Hello, i'm trying to fix a bug in django type stubs, and i believe i've stubled upon bug in mypy.
Django has following syntax for defining foreign keys:
class Book(models.Model):
author = models.ForeignKey(Author, null=True)
This means author should be Author | None.
Right now django-types are using @override for __init__ with differently typed self parameter:
@overload
def __init__(
self: ForeignKey[_M],
...
null: Literal[False],
...
@overload
def __init__(
self: ForeignKey[Optional[_M]],
...
null: Literal[True],
...
And it works for mypy, but doesn't work for pyright. In mypy it works becouse it uses undocumented feature of typing self, as i discovered here:
Proper, and most concise way of doing this, would be overloading __new__ with args/kwargs and bool: Literal[...], and leaving full annotation with all arguments typed in __init__ (which i attempted here https://github.com/sbdchd/django-types/pull/86).
But, when we try to do this: mypy would ignore overrides in __new__ and will always use non-optional case.
To Reproduce
To reproduce this without much clutter, we can use following code:
# related_field_minimal.pyi
from typing import Generic, Literal, Optional, Type, TypeVar, overload
class Model: ...
_M = TypeVar("_M", bound=Optional[Model])
class ForeignKey(Generic[_M]):
@overload
def __new__(
cls,
to: Type[_M],
null: Literal[False] = ...,
) -> ForeignKey[_M]: ...
@overload
def __new__(
cls,
to: Type[_M],
null: Literal[True],
) -> ForeignKey[Optional[_M]]: ...
def __init__(self, to: Type[_M], null: bool) -> None: ...
# models.py
from typing import Optional
from related_field_minimal import ForeignKey, Model
class Test(Model):
pass
class BookModel(Model):
required_field = ForeignKey(to=Test, null=False)
optional_f1 = ForeignKey(to=Test, null=True)
optional_f2 = ForeignKey[Test](to=Test, null=True)
optional_f3 = ForeignKey[Optional[Test]](to=Test, null=True)
reveal_type(BookModel.required_field)
reveal_type(BookModel.optional_f1)
reveal_type(BookModel.optional_f2)
reveal_type(BookModel.optional_f3)
When there is no __init__ in ForeignKey, inferred types are correct, both in mypy and pyright.
models.py:15: note: Revealed type is "related_field_minimal.ForeignKey[models.Test]"
models.py:16: note: Revealed type is "related_field_minimal.ForeignKey[Union[models.Test, None]]"
models.py:17: note: Revealed type is "related_field_minimal.ForeignKey[Union[models.Test, None]]"
models.py:18: note: Revealed type is "related_field_minimal.ForeignKey[Union[models.Test, None]]"
As soon as we add __init__ to our type definition in related_field_minimal.pyi, it becomes incorrect in mypy:
models.py:15: note: Revealed type is "related_field_minimal.ForeignKey[models.Test]"
models.py:16: note: Revealed type is "related_field_minimal.ForeignKey[models.Test]"
models.py:17: note: Revealed type is "related_field_minimal.ForeignKey[models.Test]"
models.py:18: note: Revealed type is "related_field_minimal.ForeignKey[Union[models.Test, None]]"
(pyright still handles this case correctly).
Your Environment
- Mypy version used: 0.931
- Mypy command-line flags: None
- Mypy configuration options from
mypy.ini(and other config files): None - Python version used: 3.10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the mypy 0.931 reproduction using related_field_minimal.pyi and models.py, comparing results with and without init. Check the revealed types against the expected ForeignKey[Test] and ForeignKey[Optional[Test]] outputs; the issue is done when the overloads remain correct after adding the fully typed init.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100