indygreg / indygreg/PyOxidizer

Windows: Make python3.dll optional to include with standalone_dynamic windows distribution

Open
#359 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

**UPDATED** 1/21/2021 to reflect my new understanding that `python3.dll` is linked to by some extension modules to access [Python's stable ABI](https://docs.python.org/3/c-api/stable.html).

The default `standalone_dynamic` flavor of Python distribution on Windows with MSVC does not link with `python3.dll`. To see this, create a minimal standalone-dynamic executable:

```bzl
def make():
dist = default_python_distribution(flavor="standalone_dynamic")
files = FileManifest()
files.add_python_resource(
".",
dist.to_python_executable(
name="pyapp",
packaging_policy=dist.make_python_packaging_policy(),
config=dist.make_python_interpreter_config(),
))
return files

register_target("install", make, default=True)
resolve_targets()
```
After runnning `pyoxidizer build`, `\build\x86_64-pc-windows-msvc\debug\install` now contains `lib`, `pyapp.exe`, `python38.dll`, and `python3.dll`. Now we can use [`dumpbin`](https://docs.microsoft.com/en-us/cpp/build/reference/dumpbin-reference) with [`/dependents`](https://docs.microsoft.com/en-us/cpp/build/reference/dependents) to verify that `python3.dll` is not actually used. Change into that directory and run

```console
> dumpbin pyapp.exe /dependents
Microsoft (R) COFF/PE Dumper Version 14.26.28806.0
Copyright (C) Microsoft Corporation. All rights reserved.

Dump of file pyapp.exe

File Type: EXECUTABLE IMAGE

Image has the following dependencies:

ADVAPI32.dll
KERNEL32.dll
python38.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
api-ms-win-crt-utility-l1-1-0.dll
api-ms-win-crt-convert-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-math-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-locale-l1-1-0.dll

Summary

3000 .data
47000 .pdata
1ABD000 .rdata
2000 .reloc
1000 .rsrc
344000 .text
```
Repeat the above experiment replacing `pyapp.exe` with `python38.dll` and the wildcard `lib\*.pyd`. None of them load `python3.dll`.

Because `python3.dll` is not needed by default, it would be nice to have an option removing it and some documentation pointing at [PEP 384](https://www.python.org/dev/peps/pep-0384/#linkage), which explains that `python3.dll` is for the stable ABI. This would make it easier to de-clutter an installation, and it hints to users of PyOxidizer how to start dealing with DLL availability/compatibility issues.

(Cf. #353 requesting that PyOxidizer place `vcruntime140.dll` next to the `.exe`.)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the standalone_dynamic Windows distribution example and use dumpbin /dependents on pyapp.exe, python38.dll, and lib\*.pyd to confirm current dependencies. Read the packaging and distribution behavior around python3.dll and PEP 384's stable-ABI linkage. Done means python3.dll can be omitted through an explicit option and the relevant documentation explains when it is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
build-system, operating-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.