indygreg / indygreg/PyOxidizer

`importlib.import_module` raising ModuleNotFoundError with custom sys.path

Open
#664 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

Hello! I'm [porting](https://github.com/elastic/rally/pull/1637) Rally (the official Elasticsearch benchmark tool) to PyOxidizer, and despite being stuck on 0.22.0 it works really well and required almost no change to our code. However Rally has this feature where it will load custom Python modules from another directory, for example https://github.com/elastic/rally-tracks/blob/master/eql/track.py.

Suppose `/home/q/src/pyapp/other/path/amodule.py` exists. The following CPython code does not raise an exception and loads the module:

```python3
import importlib
import sys

sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")
```

But translated to PyOxidizer, I get an exception. Here's the `pyoxidizer.bzl` file that attempts to load the same module:

```python3
def make_exe():
dist = default_python_distribution()
policy = dist.make_python_packaging_policy()
policy.resources_location = "in-memory"

python_config = dist.make_python_interpreter_config()
# Evaluate a string as Python code when the interpreter starts.
python_config.run_command = """
import importlib
import sys

sys.path.insert(0, "/home/q/src/pyapp/other/path")
importlib.import_module("amodule")
"""

return dist.to_python_executable(
name="pyapp",
packaging_policy=policy,
config=python_config
)

def make_install(exe):
files = FileManifest()
files.add_python_resource(".", exe)
return files

register_target("exe", make_exe)
register_target("install", make_install, depends=["exe"], default=True)

resolve_targets()
```

But `pyoxidizer run` fails with:

```python3
installing files to /home/q/src/pyapp/./build/x86_64-unknown-linux-gnu/debug/install
Traceback (most recent call last):
File "", line 4, in
File "importlib", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'amodule'
error: cargo run failed
```

I would appreciate any ideas to help me debug this, or even turns this into a reproducer using only `oxidizer_importer`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the pyoxidizer.bzl example and run_command that inserts /home/q/src/pyapp/other/path before calling importlib.import_module("amodule"). Compare this behavior with the working CPython example, then try reducing it to a reproducer using oxidizer_importer as requested. Done means the cause of the ModuleNotFoundError is identified and a minimal reproducer or resolution is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.