__new__ type annotations have unexpected behavior in some cases

Bug Report

I am trying to emulate some Pandas type behavior, where Index.__new__ can return different types. The simplified case I am trying to model is that sequences of np.datetime64 and datetime.datetime get turned into one class, and other objects get turned into another class.

The following is the only way I've gotten it to work, after trying many many variations:

# THIS VERSION WORKS, BUT REQUIRES MODIFYING CODE WITH EXTRA CLASS
from typing import TypeVar, Generic, List, Union, overload
from typing_extensions import Protocol
from datetime import datetime

T = TypeVar("T", covariant=True)
S = TypeVar("S")

class datetime64(int):
    """Stand-in for np.datetime64."""


class IndexType(Protocol[T]):
    def first(self) -> T: ...


class Index:

    @overload
    def __new__(cls, values: List[datetime64]) -> "Datetime64Index": ...
    @overload
    def __new__(cls, values: List[datetime]) -> "Datetime64Index": ...
    @overload
    def __new__(cls, values: List[S]) -> "DefaultIndex[S]": ...

    def __new__(cls, values):
        if type(values[0]) in (datetime, datetime64):
            cls = Datetime64Index
        else:
            cls = DefaultIndex
        return object.__new__(cls)


class DefaultIndex(Index, Generic[S]):
    def __init__(self, values: List[S]):
        self.values = values

    def first(self) -> S:
        return self.values[0]


class Datetime64Index(DefaultIndex):

    def __init__(self, values: Union[List[datetime], List[datetime64]]):
        self.values : List[datetime64] = [
            datetime64(o.timestamp()) if isinstance(o, datetime) else o
            for o in values
        ]

    def first(self) -> datetime:
        return datetime.fromtimestamp(self.values[0])


# Should work
a: IndexType[datetime] = Index([datetime64(100)])
b: IndexType[datetime] = Index([datetime(2000, 10, 20)])
c: IndexType[bool] = Index([True])

# Should complain
d: IndexType[datetime] = Index(["a"])
e: IndexType[bool] = Index(["a"])

As expected, mypy only complains about the last two lines:

$ mypy test.py
test.py:59: error: List item 0 has incompatible type "str"; expected "datetime64"
test.py:60: error: Incompatible types in assignment (expression has type "Datetime64Index", variable has type "IndexType[bool]")
test.py:60: note: Following member(s) of "Datetime64Index" have conflicts:
test.py:60: note:     Expected:
test.py:60: note:         def first(self) -> bool
test.py:60: note:     Got:
test.py:60: note:         def first(self) -> datetime
test.py:60: error: List item 0 has incompatible type "str"; expected "datetime64"

However, the need for DefaultIndex feels like a hack. What I would actually like to do is the following:

# THIS VERSION SHOULD WORK, BUT CAUSES MYPY TO ERRONEOUSLY(?) COMPLAIN
from typing import TypeVar, Generic, List, Union, overload
from typing_extensions import Protocol
from datetime import datetime

T = TypeVar("T", covariant=True)
S = TypeVar("S")

class datetime64(int):
    """Stand-in for np.datetime64."""


class IndexType(Protocol[T]):
    def first(self) -> T: ...


class Index(Generic[S]):

    @overload
    def __new__(cls, values: List[datetime64]) -> "Datetime64Index": ...
    @overload
    def __new__(cls, values: List[datetime]) -> "Datetime64Index": ...
    @overload
    def __new__(cls, values: List[S]) -> "Index[S]": ...

    def __new__(cls, values):
        if type(values[0]) in (datetime, datetime64):
            cls = Datetime64Index
        return object.__new__(cls)

    def __init__(self, values: List[S]):
        self.values = values

    def first(self) -> S:
        return self.values[0]


class Datetime64Index(Index):

    def __init__(self, values: Union[List[datetime], List[datetime64]]):
        self.values : List[datetime64] = [
            datetime64(o.timestamp()) if isinstance(o, datetime) else o
            for o in values
        ]

    def first(self) -> datetime:
        return datetime.fromtimestamp(self.values[0])


# Should work
a: IndexType[datetime] = Index([datetime64(100)])
b: IndexType[datetime] = Index([datetime(2000, 10, 20)])
c: IndexType[bool] = Index([True])

# Should complain
d: IndexType[datetime] = Index(["a"])
e: IndexType[bool] = Index(["a"])

However, mypy gets confused and complains about a: IndexType[datetime] = Index([datetime64(100)]):

test.py:50: error: List item 0 has incompatible type "datetime64"; expected "datetime"
test.py:55: error: List item 0 has incompatible type "str"; expected "datetime"
test.py:56: error: List item 0 has incompatible type "str"; expected "bool"
Found 3 errors in 1 file (checked 1 source file)

Your Environment

• Mypy version used: 0.782
• Mypy command-line flags: None
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: 3.7
• Operating system and version: Linux

Reproduce the reported diagnostics with the supplied test.py examples using mypy 0.782 and no configuration. Start by tracing overload resolution and generic new handling for Index, then compare the inferred types and diagnostics with the expected assignments; done means the datetime64, datetime, bool, and invalid string cases are checked correctly.