JetBrains / JetBrains/benjamin-plus-skill
find_spec probe is not robust for dotted module names
- Dominant language
- Shell
- Stars
- 321
- Forks
- 14
- Avg merge
- 9h 10m
- Merged PRs (30d)
- 2
Description
## Summary
Follow-up to #5.
The updated Python dependency-probe example now uses `importlib.util.find_spec()`:
```bash
python3 -c "import importlib.util as u; [print(m) for m in ['x','y','z'] if not u.find_spec(m)]"
```
This fixes the original `import x, y, z` problem for simple top-level module
names, but dotted module names have two important behaviors:
1. if the parent package is missing, `find_spec("parent.child")` can raise
`ModuleNotFoundError` and abort the entire batch;
2. if the parent exists, resolving the child imports the parent package, so the
probe is not necessarily side-effect-free.
Dotted module names are common in real Python code: `django.conf`,
`sqlalchemy.orm`, `google.cloud.storage`, `PIL.Image`, `matplotlib.pyplot`, etc.
Python's own documentation notes this behavior explicitly: when looking up a
submodule, `find_spec()` imports the parent module.
https://docs.python.org/3/library/importlib.html#importlib.util.find_spec
## Real-world failure case: missing Django dependency
A fresh virtual environment without Django reproduces the first problem with a
real module name:
```bash
python3 -m venv /tmp/bp-findspec-venv
/tmp/bp-findspec-venv/bin/python - <<'PY'
import importlib.util as u
mods = [
"django.conf",
"json",
"definitely_missing_other",
]
[print(m) for m in mods if not u.find_spec(m)]
print("completed")
PY
```
Observed result:
```text
Traceback (most recent call last):
...
ModuleNotFoundError: No module named 'django'
```
The probe aborts on `django.conf`.
It never checks `json`, never reports `definitely_missing_other`, and never
prints `completed`.
This is exactly the dependency-probe scenario: a project contains an import such
as:
```python
from django.conf import settings
```
but Django is not installed yet.
If an agent derives probe targets from actual imports and uses `django.conf`,
the updated one-pass probe can still fall back to discovering dependencies one
failure at a time.
## Real-world side effect: SQLAlchemy parent import
With SQLAlchemy installed:
```bash
python3 - <<'PY'
import importlib.util as u
import sys
before = set(sys.modules)
print("sqlalchemy before:", "sqlalchemy" in sys.modules)
spec = u.find_spec("sqlalchemy.orm")
after = set(sys.modules)
print("spec found:", bool(spec))
print("sqlalchemy after:", "sqlalchemy" in sys.modules)
print("sqlalchemy.orm after:", "sqlalchemy.orm" in sys.modules)
print("new modules loaded:", len(after - before))
PY
```
In the environment where I tested this (Python 3.13.5, current SQLAlchemy
available in the environment), the result was:
```text
sqlalchemy before: False
spec found: True
sqlalchemy after: True
sqlalchemy.orm after: False
new modules loaded: 116
```
The exact number of transitive modules is version/environment dependent, but the
important part is stable: asking for the spec of `sqlalchemy.orm` imported the
parent `sqlalchemy` package.
For packages with meaningful `__init__.py` behavior this can mean startup work,
configuration reads, warnings/output, native-library initialization, or even an
exception. So `find_spec()` is not a pure existence check for arbitrary dotted
module names.
## Scope
This does **not** make `find_spec()` a bad replacement for the original example.
For simple top-level import names such as:
```text
requests
numpy
pytest
PIL
```
it is a useful cheap probe.
The narrower problem is that the rule currently presents the expression as a
generic way to "check them all in one probe", while actual Python imports often
contain dotted names.
## Possible adjustment
The smallest fix may be to scope the example explicitly to **top-level import
packages**:
> Derive the top-level import package for each dependency and probe those with
> `find_spec()`.
For example:
```text
django.conf -> django
sqlalchemy.orm -> sqlalchemy
google.cloud.storage -> google
PIL.Image -> PIL
```
If checking exact dotted module paths is actually required, probe each one
independently with exception handling and document that resolving a submodule
may import its parent.
That keeps the one-pass dependency check while avoiding both failure modes.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the updated Python dependency-probe example and its current find_spec() expression; reproduce the django.conf case in a fresh virtual environment. Done when dotted imports no longer abort the batch and the documented behavior matches the chosen handling, including the parent-import caveat.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100