python / python/mypy

Mypy does not correctly narrow indexing operations when using Literal or Final keys

Open
#7,905 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

priority-2-low topic-final topic-literal-types topic-type-narrowing topic-typed-dict
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Consider the following program:

from typing_extensions import Literal, TypedDict
from enum import Enum

class Key(Enum):
    X = 1
    Y = 2
    Z = 3

class MyDict(TypedDict):
    key: Literal[Key.X, Key.Y]
    blah: int

KEY: Literal["key"] = "key"

d: MyDict

if d["key"] is Key.X:
    reveal_type(d["key"])  # note: Revealed type is 'Literal[Key.X]'

if d[KEY] is Key.X:
    reveal_type(d[KEY])  # note: Revealed type is 'Literal[Key.X, Key.Y]'

Mypy is currently capable of narrowing expressions like d["key"], which we can see in the first expression.

So, it's natural to assume that mypy would be able to do the same for the second since the two programs are theoretically identical -- but we can't.

The root cause has to do with the "literal" subsystem (which is not to be confused with the Literal types subsystem) here: https://github.com/python/mypy/blob/master/mypy/literals.py#L65

The index in the second example is a NameExpr, which causes the if statement to evaluate to false and return a LITERAL_NO. This then makes the narrowing logic rule out d[KEY] as a candidate for narrowing in https://github.com/python/mypy/blob/master/mypy/checker.py#L3775.

I'm not really sure what the best way of fixing this would be. The natural solution would be to also pass along the expression type into the literal(...) function, but that seems very annoying to do. I'm also not entirely sure whether this is even a sound narrowing: I'm not very familiar with the "literals" subsystem, or how it's meant to interact with Literal types.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction in the issue, then inspect mypy/literals.py around literal() and mypy/checker.py around the narrowing logic at line 3775. Trace why a NameExpr for KEY produces LITERAL_NO and compare it with the direct string index. Done when the KEY case narrows d[KEY] to Literal[Key.X] consistently with d["key"].

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.