indygreg / indygreg/PyOxidizer

Migrating from `importlib.resources.open_text` to `files` looks broken

Open
#529 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

First of all, let me say the pyoxidizer works great! I have a pure python project where I use pyoxidizer to build a fully standalone executable (no issues here).

I wanted to get rid of some warnings in that Python project and migrate from `importlib.resources.open_text` to the `files` API, as described [here](https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy).

It looks like this breaks pyoxidizer:

```
File "importlib.resources", line 147, in files
File "importlib._common", line 14, in from_package
File "importlib._common", line 18, in fallback_resources
File "pathlib", line 1082, in __new__
File "pathlib", line 707, in _from_parts
File "pathlib", line 691, in _parse_args
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

After digging a bit, it looks like the `files()` as implemented [here](https://github.com/python/cpython/blob/3.9/Lib/importlib/resources.py#L143-L147) relies on `fallback_resources` as implemented [there](https://github.com/python/cpython/blob/3.9/Lib/importlib/_common.py#L9-L25).
The python modules returned by `import_module` from `oxidized_importer.OxidizedFinder`, seem to have their `__spec__.origin` set to None. This breaks the `fallback_resources`, that is never called when using [open_text](https://github.com/python/cpython/blob/3.9/Lib/importlib/resources.py#L85-L121) (there seems to be a fallback on empty `__spec__.origin` in that code path).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at oxidized_importer.OxidizedFinder and inspect how imported modules receive __spec__.origin, then trace the importlib.resources.files and fallback_resources path described in the issue. Reproduce the failure in a standalone PyOxidizer executable using the modern files API; done means resource loading works without the NoneType error while the existing open_text path remains functional.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.