indygreg / indygreg/PyOxidizer
Windows: "unable to load extension module library from memory" for sqlite3
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
On Windows 10 (version 19042.572) with PyOxidizer 0.9.0 (rustc 1.47.0 (18bf6b4f0 2020-10-07) stable-x86_64-pc-windows-msvc), place the following in `pyoxidizer.bzl` in an otherwise empty directory.
```bzl
def make():
dist = default_python_distribution(flavor="standalone_dynamic")
python_config = dist.make_python_interpreter_config()
python_config.config_profile = "python"
python_config.run_mode = "eval:import sqlite3"
policy = dist.make_python_packaging_policy()
policy.set_resource_handling_mode("classify")
policy.resources_location = "in-memory"
policy.allow_in_memory_shared_library_loading = True
exe = dist.to_python_executable(
name="pyapp",
packaging_policy=policy,
config=python_config,
)
files = FileManifest()
files.add_python_resource(".", exe)
return files
register_target("install", make, default=True)
resolve_targets()
```
Executing `pyoxidizer run` gives the following output (build output omitted):
```python-traceback
Traceback (most recent call last):
File "", line 1, in
File "sqlite3", line 23, in
File "sqlite3.dbapi2", line 27, in
ImportError: ('unable to load extension module library from memory', '_sqlite3')
```
The error goes away if you make the following change to `pyoxidizer.bzl`:
```diff
- policy.allow_in_memory_shared_library_loading = True
+ policy.resources_location_fallback = "filesystem-relative:Lib"
+ policy.allow_in_memory_shared_library_loading = False
```
Granted, the docs for [`PythonPackagingPolicy.allow_in_memory_shared_library_loading`](https://pyoxidizer.readthedocs.io/en/stable/config_type_python_packaging_policy.html#config-type-python-packaging-policy-allow-in-memory-shared-library-loading) do say (emphasis mine):
> Some platforms (notably Windows) allow opening shared libraries from a memory address. This mode of opening shared libraries allows libraries to be embedded in binaries without having to statically link them. However, **not every library works correctly when loaded this way.**
So I can't tell if this error is documented behavior or not. At the very least it would be helpful for the documentation to say how to predict what will work and what won't. But it'd also be nice if it worked...
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the Windows failure using the minimal pyoxidizer.bzl configuration and run pyoxidizer run with sqlite3 imported. Compare it with the filesystem-relative fallback, then determine whether the in-memory loading behavior needs a fix or clearer documentation describing sqlite3 compatibility. Done means the failure is resolved or the documented limitation and workaround are precise.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust, sqlite
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100