micropython / micropython/micropython

mpy-tool does not properly collect module names for imports

Open
#7,132 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-info
Dominant language
C
Stars
22.1k
Forks
9k
Avg merge
6d 4h
Merged PRs (30d)
16

Description

Consider an application that has the following module file:
foo/bar/quux.py

Put an import statement in some other file:

from foo.bar import quux

When freezing the module with mpy-tool.py, the names foo.bar and quux are collected. However:

  • foo.bar.quux is not collected, which will be created as a qstr at runtime, because sys.modules uses it as a key
  • neither foo nor bar is collected. bar will be created as a qstr at runtime, because the dict of foo needs to insert it as a key

Of course, if we use relative imports, there's a similar problem: in foo/bar/baz.py, from . import quux should (but can't really know to) also collect foo.bar.quux despite the string not existing anywhere in the file.

I'm not sure if this is something to solve in mpy-tool itself, or whether there should be a separate step that collects symbols in this way. But it seems that using file names to generate both the fully qualified module name and the individual components would be the right thing to do. (that is basically the workaround i'm using: generate all_modules.py that walk the filesystem and convert every py file name to import that.file.name, which collects "that.file.name", and that.file.name which collects "that"", "file" and "name")

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading mpy-tool.py, focusing on how import statements and module names are collected. Reproduce the foo/bar/quux.py and from foo.bar import quux example, then compare the collected names with the qstrs needed at runtime. Done means the fully qualified module name and its individual components are collected, with the relative-import limitation considered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.