python / python/mypy

Irregularities in populating new dict from a source dict with a narrower key type

Open
#16,557 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

When creating or updating a new dict with items from another dict with a narrower key type, Mypy complains about some methods which are actually perfectly valid.

To Reproduce

from typing import TypeVar

_KT = TypeVar("_KT")
_VT = TypeVar("_VT")


def clone_dict(source_dict: dict[_KT, _VT]) -> dict[_KT, _VT]:
    return dict(source_dict)


def test(source_dict: dict[str, None]) -> list[dict[str | int, None]]:
    a: dict[str | int, None] = {"a": None, "b": None}
    b: dict[str | int, None] = {k: v for k, v in source_dict.items()}
    c: dict[str | int, None] = {k: None for k in source_dict.keys()}
    d: dict[str | int, None] = {**source_dict}
    e: dict[str | int, None] = dict(source_dict.items())
    f: dict[str | int, None] = dict(source_dict)
    g: dict[str | int, None] = source_dict.copy()
    h: dict[str | int, None] = clone_dict(source_dict)
    i: dict[str | int, None] = {}
    i.update(source_dict.items())
    j: dict[str | int, None] = {}
    j.update(source_dict)

    return [a, b, c, d, e, f, g, h, i, j]

Expected Behavior

The above code shows 10 ways of creating the new dictionary:

  • a doesn't actually clone the source dict; it just creates a new dict from a dict literal that just happens to only contain string keys
  • b copies the source dict using a dict comprehension and .items()
  • c uses a dict comprehension and .keys() (possible in this case because the dict value type is None, which is a singleton)
  • d uses a dict comprehension and star-star-expansion
  • e uses the dict() constructor and .items()
  • f uses the dict() constructor and the source dict directly
  • g uses the built-in .copy() method
  • h uses a user-created dict copying method
  • i creates an empty dict and then .update()s it using .items()
  • j creates an empty dict and then .update()s it using the source dict directly

Of these:

  • Mypy should complain about h; Mypy has no way of knowing that the function in question is returning a fresh dict which is used nowhere else (and thus can safely be implicitly cast to a wider type)
  • Mypy might complain about g; since that is a built-in method, it could be special-cased into Mypy that this is safe, but absent such a special case it would have the same semantics as clone_dict()
  • All the other option are safe and should not raise any error

Actual Behavior

Mypy correctly warns about g and h, but also issues spurious warnings about d, f, and j:

$ mypy test.py
test.py:15: error: Unpacked dict entry 0 has incompatible type "dict[str, None]"; expected "SupportsKeysAndGetItem[str | int, None]"  [dict-item]
test.py:17: error: Incompatible types in assignment (expression has type "dict[str, None]", variable has type "dict[str | int, None]")  [assignment]
test.py:18: error: Incompatible types in assignment (expression has type "dict[str, None]", variable has type "dict[str | int, None]")  [assignment]
test.py:19: error: Argument 1 to "clone_dict" has incompatible type "dict[str, None]"; expected "dict[str | int, None]"  [arg-type]
test.py:23: error: Argument 1 to "update" of "MutableMapping" has incompatible type "dict[str, None]"; expected "SupportsKeysAndGetItem[str | int, None]"  [arg-type]
Found 5 errors in 1 file (checked 1 source file)

Your Environment

  • Mypy version used: mypy 1.5.1 (compiled: yes)
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: Python 3.11.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided reproducer with mypy and compare diagnostics for dict unpacking, the dict constructor, and update(). Trace the type-checking paths for those operations, then confirm that the safe cases no longer report errors while the clone_dict() and copy() cases retain their expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.