python / python/mypy

mypy doesn't recognize intra-package imports when qualified by `__all__`

Open
#10,826 3 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

I have a package and am trying to control which classes/functions are available at the 'top level' of the package. For this purpose, I am using the __all__ dunder and star imports in the __init__.py files. I get unexpected behaviour from mypy when doing this: some of my objects are "not defined", when they in fact are.

To Reproduce

This bug has to do with imports between files, so to reproduce this, we need a bit of a package file structure. Consider the package gym:

gym
  sup/
    __init__.py
    bro.py
  __init__.py
  cool.py

File contents would be:

# sup/bro.py - This is the object we want to be available at the top level
__all__ = ["Bro"]

class Bro:
    pass
# sup/__init__.py - This brings the Bro object to the sup module level
from . import bro
from .bro import *

__all__ = bro.__all__
# __init__.py - This brings the sup module, including the Bro object, to the top module level
from .sup import *
# cool.py - This file uses the Bro object in a function
import gym

__all__ = ["story"]

def story() -> gym.Bro:
    return gym.Bro()

Now, when I run mypy gym/ I get the following errors:

gym/cool.py:6: error: Name "gym.Bro" is not defined
gym/cool.py:7: error: Module has no attribute "Bro"
Found 2 errors in 1 file (checked 4 source files)

However, when I remove the __all__ dunder from the sup/__init__.py file, everything works as expected, and I get no errors! Unfortunately this has the unwanted side-effect that the submodule gym.sup.bro becomes available as gym.bro, which I do not want.

Somehow that __all__ dunder stops mypy from recognizing that the Bro object should arrive at the module level above it. I can also work around the error by using the full path to the Bro object, gym.sup.bro.Bro, or even gym.sup.Bro, but not gym.Bro. This unfortunately gets very verbose in larger packages.

What's going on here?

Your Environment

  • Mypy version used: 0.910
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.9.5
  • Operating system and version: WSL Ubuntu 20

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the listed gym package files and run mypy gym/. Start by tracing how all and star imports in sup/init.py are handled when resolving gym.Bro from cool.py. Done means gym.Bro is recognized without exposing gym.bro, with the reproduction passing without errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.