python / python/mypy

Return types for Overloads with "Optional[TypeVar]" arguments are deducted as just "object"

Open
#16,591 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

Please see the following mypy playground: https://mypy-play.net/?mypy=latest&python=3.12&gist=207fe333eb8e226d36b84bf34be5ca65

I personally encountered this on mypy 1.7.1 and Python 3.8, but the mypy playground link uses the latest version of everything.

Consider the following helper function:

import datetime
from typing import Any, Optional, Sized, Tuple, Type, TypeVar, Union

from typing_extensions import Literal, overload, TypeAlias

_T = TypeVar("_T")


class TypeGuardError(Exception):
    pass


_THasLen = TypeVar("_THasLen", bound=Sized)


@overload
def required(
    value: Optional[_T],
    *,
    message: Optional[str] = None,
    property_name: Optional[str] = None,
) -> _T:
    pass


@overload
def required(
    value: Any,
    *,
    instance_of: Type[_T],
    instance_of_message: Optional[str] = None,
    message: Optional[str] = None,
    property_name: Optional[str] = None,
) -> _T:
    pass


def required(
    value: Optional[Any],
    *,
    instance_of: Optional[Any] = None,
    message: Optional[str] = None,
    property_name: Optional[str] = None,
    instance_of_message: Optional[str] = None,
) -> Any:
    if property_name is not None:
        message_prefix = f"{property_name}: "
    else:
        message_prefix = ""

    if value is None:
        raise TypeGuardError(message or f"{message_prefix}Value cannot be `None`.")

    if instance_of and not isinstance(value, instance_of):
        raise TypeGuardError(
            instance_of_message
            or message
            or f'{message_prefix}Expected value of type "{instance_of.__name__}", got value of type "{type(value).__name__}" instead.'
        )

    return value

required here just checks if the value is not None, optionally verifying its type as well. If any validation fails, it's able to generate an error, with either a message you define, or a parameterizable auto-generated one.

The returned value is just the input value, but cast to its "pure type" through overloading (removing None from the type by type-matching it to Optional). But when passing the instance_of argument, the return type is whatever type you pass to that argument instead. That's the intention.

However, mypy fails my expectations. For example:

TimeDeltaLike: TypeAlias = Union[int, float, str, datetime.timedelta]

default_lock_time: Optional[TimeDeltaLike] = None

locked_for = required(
    default_lock_time,
    message=f"Entity does not specify a default lock time.",
)

# QUESTION: Why is the revealed type `object`?
reveal_type(locked_for)

In the last line, the revealed type is object.

When analyzing the same code through Pyright, the revealed type is int | float | str | datetime.timedelta, as I would expect.

To Reproduce

# Ideally, a small sample program that demonstrates the problem.
# Or even better, a reproducible playground link https://mypy-play.net/ (use the "Gist" button)

See explanation above.

Expected Behavior

See explanation above.

Actual Behavior

See explanation above.

Your Environment

  • Mypy version used: 1.7.1
  • Mypy command-line flags:
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used: 3.8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked mypy playground and the required overloads using Optional[TypeVar]; inspect how the call with default_lock_time is inferred before reveal_type. The work is done when the revealed return type is int | float | str | datetime.timedelta rather than object, while preserving the instance_of overload behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.