indygreg / indygreg/PyOxidizer
`importlib.import_module` raising ModuleNotFoundError with custom sys.path
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
Hello! I'm [porting](https://github.com/elastic/rally/pull/1637) Rally (the official Elasticsearch benchmark tool) to PyOxidizer, and despite being stuck on 0.22.0 it works really well and required almost no change to our code. However Rally has this feature where it will load custom Python modules from another directory, for example https://github.com/elastic/rally-tracks/blob/master/eql/track.py.
Suppose `/home/q/src/pyapp/other/path/amodule.py` exists. The following CPython code does not raise an exception and loads the module:
```python3
import importlib
import sys
sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")
```
But translated to PyOxidizer, I get an exception. Here's the `pyoxidizer.bzl` file that attempts to load the same module:
```python3
def make_exe():
dist = default_python_distribution()
policy = dist.make_python_packaging_policy()
policy.resources_location = "in-memory"
python_config = dist.make_python_interpreter_config()
# Evaluate a string as Python code when the interpreter starts.
python_config.run_command = """
import importlib
import sys
sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")
"""
return dist.to_python_executable(
name="pyapp",
packaging_policy=policy,
config=python_config
)
def make_install(exe):
files = FileManifest()
files.add_python_resource(".", exe)
return files
register_target("exe", make_exe)
register_target("install", make_install, depends=["exe"], default=True)
resolve_targets()
```
But `pyoxidizer run` fails with:
```python3
installing files to /home/q/src/pyapp/./build/x86_64-unknown-linux-gnu/debug/install
Traceback (most recent call last):
File "", line 4, in
File "importlib", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'amodule'
error: cargo run failed
```
I would appreciate any ideas to help me debug this, or even turns this into a reproducer using only `oxidizer_importer`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the pyoxidizer.bzl example and run_command that inserts /home/q/src/pyapp/other/path before calling importlib.import_module("amodule"). Compare this behavior with the working CPython example, then try reducing it to a reproducer using oxidizer_importer as requested. Done means the cause of the ModuleNotFoundError is identified and a minimal reproducer or resolution is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100