indygreg / indygreg/PyOxidizer
[help/bug?] dynamic importing of python modules from the file system
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
We have a test application running `pytest` that uses some pip packages and some rust code.
The test writers develop mainly on the pytest tests, but rarely change the rust code and python packages. To keep iteration time low, I'm bundling the python interpreter, pip packages and rust code using pyoxidizer, but allowing users to run their pytest tests from the file system. Our build system scans the file system for changes, and then determines whether a rebuild of the executable (rust or pip changes), or just a re-run of the tests (pytest code changes) is necessary.
Pytest internally uses the (py)[https://github.com/pytest-dev/py] pacakge to dynamically import the test modules from the file system. It first finds the test file, adds the directory to the path, then uses `__import__` to load the module. (see https://github.com/pytest-dev/py/blob/master/py/_path/local.py#L701)
This breaks when using pyoxidizer, as it can't find the module, even if sys.path is correct. I've tried rewriting it to use `importlib.load_module` but that also fails.
I have locally modified copy of `py` that uses the code below instead of `__import__`, but I'm wondering if pyoxidier is intending to disallow the dynamic import of modules from the file system.
```python
spec = importlib.util.spec_from_file_location(modname, self.realpath())
new_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(new_mod)
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the filesystem import failure with the pytest scenario described, then inspect py/_path/local.py around line 701 and PyOxidizer's handling of bundled Python modules. Compare the __import__ path with the importlib.util approach shown in the issue. Done means establishing whether this import pattern is supported and identifying a documented or tested resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100