adafruit / adafruit/circup

circup install --auto hits RecursionError on packages with re-exporting __init__.py

Open
#275 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
172
Forks
41
Avg merge
1d 20h
Merged PRs (30d)
1

Description

## Summary

`circup install --auto` (v3.0.1) raises `RecursionError: maximum recursion depth exceeded` whenever the firmware's `code.py` imports a local package whose `__init__.py` re-exports symbols via `from .submodule import name` and any sub-module contains a `from ..` style import.

The recursion happens entirely inside `circup.command_utils.get_all_imports`, before any install or network activity.

## Reproducer

A minimal reproducer is attached as **`circup-recursion-repro.zip`**. [circup-recursion-repro.zip](https://github.com/user-attachments/files/27575679/circup-recursion-repro.zip) After unzipping:

```bash
pip install circup==3.0.1
circup --path=./circup-recursion-repro install --auto
```

Expected (buggy) output:

```
Finding imports from: code.py
Traceback (most recent call last):
...
File ".../circup/command_utils.py", line 888, in get_all_imports
sub_imports = get_all_imports(...)
[Previous line repeated 984 more times]
...
RecursionError: maximum recursion depth exceeded
```

The zip contains a `code.py`, a tiny `pkg/` with three files (`__init__.py` that re-exports, `a.py`, `b.py`), and a placebo `boot_out.txt` so circup accepts the `--path` as a device. No real bundle libraries are involved — the entire failure is in the import walker traversing the local files. See the included `README.md` for layout and a written walkthrough.

The codebase that surfaced this in production is a CircuitPython firmware whose `code.py` imports a `code_lib` package; `code_lib/__init__.py` re-exports ~15 symbols and a sub-package's `__init__.py` re-exports ~21. With those numbers the recursion blows on the very first invocation.

## Root cause

Two cooperating bugs in `circup/command_utils.py`, plus a third compounding issue:

**Bug 1 — `imports_from_code` emits bare-dot strings.** When parsing `from ..foo import bar`, the partial-prefix loop (lines 808–817) produces an import set containing `"."`, `".."`, `"..foo"`, `"..foo.bar"`. The bare `"."` and `".."` entries have no module name attached.

**Bug 2 — `get_all_imports` resolves bare dots into trailing-dot module names.** For `install == "."`, line 858 computes:

```python
install_module = ".".join(current_module.split(".")[:-1]) + "." + install[1:]
# ^ "parent" ^ "" (install is just ".")
# → install_module == "parent."
```

The next recursion sees the same bare `"."` again in its sub-file's import set, producing `"parent.."`, then `"parent..."`, etc. Each iteration appends another dot, the resolved string is unique every time, and `visited` never rejects it. The walker recurses until Python's recursion limit is hit.

**Bug 3 (compounding) — `visited` namespace mismatch.** `visited.add(current_module)` (line 838) tracks **absolute** module names like `"foo.common.base"`, but the loop dedup at line 848 (`if install in visited`) checks against the **raw relative** `install` string like `".base"`. Those namespaces never compare equal, so the same file is re-walked through every relative-import entry in its parent's `__init__.py`. Not unbounded on its own, but compounds the cost dramatically — even with Bug 2 fixed, a package with N re-exports per `__init__.py` is walked O(N) times more than necessary.

## Environment

- circup: 3.0.1 (PyPI)
- Python: 3.13.9
- OS: Ubuntu 24.04 (Linux)
- Backend: `DiskBackend` (`--path=` mode)

Not tested against `WebBackend`, but the bug is in the path-agnostic walker so I would expect identical behaviour.

## Proposed fix

Three additions to `get_all_imports`, all in `circup/command_utils.py`:

```diff
@@ def get_all_imports(backend, auto_file_content, auto_file_path, mod_names, current_module, visited=None):
for install in imports:
if install in visited:
continue
if install in mod_names:
requested_installs.append(install)
+ continue
+ # Skip bare "." / ".." entries that imports_from_code emits as
+ # partials of `from .. import X`. They have no module name attached,
+ # so resolving them produces trailing-dot install_module values that
+ # grow by one dot per recursion until the recursion limit is hit.
+ if install.lstrip(".") == "":
+ continue
- else:
- # relative module paths
- if install.startswith(".."):
+ if install.startswith(".."):
install_module = ".".join(current_module.split(".")[:-2])
install_module = install_module + "." + install[2:]
- elif install.startswith("."):
+ elif install.startswith("."):
install_module = ".".join(current_module.split(".")[:-1])
install_module = install_module + "." + install[1:]
- else:
+ else:
install_module = install
+ # Dedup against the resolved absolute name as well; raw relative `install`
+ # strings live in a different namespace from current_module, so without
+ # this the same submodule is re-walked through every relative-import entry
+ # in its parent's __init__.py.
+ if install_module in visited:
+ continue
# possible files for the module: .py or __init__.py (if directory)
file_name = os.path.join(*install_module.split(".")) + ".py"
...
if not exists:
file_name = os.path.join(*install_module.split("."), "__init__.py")
full_location = file_name
exists = backend.file_exists(full_location)
if not exists:
continue
install_module += ".__init__"
+ if install_module in visited:
+ continue
+ visited.add(install_module) # record before recursing so siblings see the same target
auto_file_content = backend.get_file_content(full_location)
if auto_file_content:
sub_imports = get_all_imports(
backend,
auto_file_content,
auto_file_path,
mod_names,
install_module,
visited,
)
requested_installs.extend(sub_imports)
```

The bare-dot skip is the load-bearing fix for Bug 2 — without it the walker is unbounded. The two new `install_module in visited` checks plus the early `visited.add(install_module)` close Bug 3 once Bug 2 is fixed.

I have a working monkey-patch wrapper applying these changes against vendored circup 3.0.1 in our project; happy to open a PR if useful.

Contributor guide

Open the contributing guide

Research direction

Start in circup/command_utils.py at get_all_imports and imports_from_code, then run the attached circup-recursion-repro.zip command from its README. Verify that --auto completes without RecursionError for the re-exporting package, that relative imports are not repeatedly walked, and that the existing import discovery behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.