python / python/mypy

Incorrect status of name in class def or module level, before being assigned or after being deleted.

Open
#12,818 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

There are some circumstances when mypy interprets a name (x) in a module or class scope as inner when it is outer, or vice versa. (There appears to be no such problem with function scopes.)
I use the term inner to mean that the python interpreter will resolve the name x in the scope. It it is not currently bound in the scope, it is a NameError. outer means that the interpreter will try to resolve the name x in some enclosing scope.
I have determined by trial and error the rules for when a name is inner or outer, and what value a name is resolved to. I wrote a script that applies these rules in a great variety of combinations of nested class and function scopes and uses of the name x, and verifies that the expected value of x, or NameError, is the actual result. You'll need to rename it to gentest.py in order to run it yourself (python 3.7).
x is inner when:

  1. it is not declared global or nonlocal
  2. and
    1. x is bound or deleted anywhere in a function scope.
    2. or x is currently bound in a class or module scope.

An inner x is resolved:

  • In a function f"
    • in a binding operation in the body of f this binds a new value to x.
    • elsewhere in the body of f by the current bound value if any, else a NameError.
    • in the body of a nested function or class s (at whatever level) where x is an outer name in s, which is resolved in f: likewise by the current bound value if any, else a NameError. If this occurs after executing the body of f (because s is a function which was saved and later called), it is the same as though the call to s occurred at the end of the body of f.
  • In a class C
    • in a binding operation in the body of C this binds a new value to x.
    • elsewhere in the body of C by the current bound value. Note, if there is no current bound value of x, then x is an outer name.
    • Note, x will never be resolved in any enclosing class or function scope, because the resolution rules skip over class scopes.
  • in a module:
    Same as for a class, except that the name x can also appear in a nested class or function.

An outer x is resolved in any scope, possibly restricted to functions only, resulting in a value, or not bound, or in the restricted case, no scope.
When restricted, x is resolved:

  • In a module scope, no scope.
  • Else in a class scope, by x in the next enclosing scope.
  • Else if x is declared nonlocal, by x in the next enclosing scope.
  • Else if x is declared global, no scope.
  • Else, by x in the next enclosing scope. Note, since x is outer, it is neither assigned nor deleted here.

When unrestricted, x is resolved:

  • In a module scope, by the value of builtins.x if it exists, else not bound. Note, global declarations are ignored and nonlocal declarations are syntax errors.
  • Else if x is declared nonlocal, by x in the next enclosing scope, restricted.
  • Else if x is declared global, by x in the module namespace
  • Else, as follows: Note, since x is outer, it is neither assigned nor deleted here.
    • If resolved by x in the next enclosing scope (restricted) succeeds, then by that result.
    • Otherwise, by x in the module scope.

A special situation occurs for augmented assignments like x += y in class and module scopes. The current value of x is found, then x.__iadd__(y) is called, then the result is assigned to x. The two x's can be in different scopes

One surprising thing, which I don't think is clear in the Python docs, is the value of a name in a class def which is bound or deleted in the class body, whenever there is no binding of that name during the execution of the class body. If the name is captured by some enclosed function scope, then the value may be found in searching enclosed functions then globals then builtins, whereas if the name is not captured, it skips the enclosed functions.

You may wish to include the above name resolution rules in your own documentation. I will forward it to the Python people as an issue about their documentation.

Your Environment

  • Mypy version used: 0.950
  • Python version used: 3.7

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Rename the attached gentest.py.txt script to gentest.py and run it with Python 3.7 to reproduce the documented name-resolution cases. Compare the expected results with mypy's behavior, then locate the name-resolution implementation and add regression coverage for the failing class and module-scope cases. Done means mypy reports these cases consistently with the stated rules.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.