python / python/mypy

`get_dynamic_class_hook` does not handle `defer` properly

Open
#17,402 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-plugins
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

typically, ctx.api.defer() can be called in a plugin when there is insufficient information to defer calling to a future pass. this "works" for get_dynamic_class_hook but causes the dynamic class assignment to be converted into a Var rendering it useless as a base class later.

I've adapted a test plugin from the mypy codebase to demonstrate this problem

To Reproduce

the plugin I've adapted responds to an EAGER=1 environment variable -- where it will not defer the class creation. EAGER=0 simulates when insufficient information is available so it must defer the class to a separate pass

# mypy.ini
[mypy]
plugins = _plugin
# _plugin.py
from __future__ import annotations

import os
from typing import Callable

from mypy.nodes import GDEF, Block, ClassDef, SymbolTable, SymbolTableNode, TypeInfo, Var
from mypy.plugin import ClassDefContext, DynamicClassDefContext, Plugin
from mypy.types import Instance, get_proper_type


_DEFERRED = os.environ.get('EAGER') == '1'


class DynPlugin(Plugin):
    def get_dynamic_class_hook(
        self, fullname: str
    ) -> Callable[[DynamicClassDefContext], None] | None:
        if fullname == "mod.declarative_base":
            return add_info_hook
        return None


def add_info_hook(ctx: DynamicClassDefContext) -> None:
    global _DEFERRED
    if not _DEFERRED:
        _DEFERRED = True
        ctx.api.defer()
        print('defering!')
        return

    class_def = ClassDef(ctx.name, Block([]))
    class_def.fullname = ctx.api.qualified_name(ctx.name)

    info = TypeInfo(SymbolTable(), class_def, ctx.api.cur_mod_id)
    class_def.info = info
    obj = ctx.api.named_type("builtins.object")
    info.mro = [info, obj.type]
    info.bases = [obj]
    ctx.api.add_symbol_table_node(ctx.name, SymbolTableNode(GDEF, info))
    print('creating!')


def plugin(version: str) -> type[DynPlugin]:
    return DynPlugin
# mod.py
def declarative_base(name: str) -> object:
    raise NotImplementedError

cls = declarative_base("wat")

class C(cls): pass

Expected Behavior

both of these should pass type checking

rm -rf .mypy_cache && EAGER=1 python -m mypy mod.py
rm -rf .mypy_cache && EAGER=0 python -m mypy mod.py

Actual Behavior

$ rm -rf .mypy_cache && EAGER=1 python -m mypy mod.py
creating!
Success: no issues found in 1 source file
$ rm -rf .mypy_cache && EAGER=0 python -m mypy mod.py
defering!
creating!
mod.py:6: error: Variable "mod.cls" is not valid as a type  [valid-type]
mod.py:6: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
mod.py:6: error: Invalid base class "cls"  [misc]
Found 2 errors in 1 file (checked 1 source file)

Your Environment

  • Mypy version used: current HEAD 10f18a82b612b6127659cd64aa60c10b9cc7a904
  • Mypy command-line flags: see above
  • Mypy configuration options from mypy.ini (and other config files): see above
  • Python version used: 3.12.2 (though it doesn't seem to matter)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with _plugin.py and mod.py using the two EAGER commands, then trace the get_dynamic_class_hook handling around ctx.api.defer(). Done means both deferred and eager runs type-check successfully without treating cls as a Var or reporting invalid base-class errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.