indygreg / indygreg/PyOxidizer

pyembed: TEST: fork tests can't allocate memory on Windows

Open
#349 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

On `main` as of 2b4e6640, 34 of pyembed's 41 Rust tests fail on Windows (10 pro, version 20H2, build 19042.630). It looks like memory allocation is broken for Python interpreters running in subprocesses started via rusty-fork. Picking one of the failed tests at random:

```
---- test::interpreter_config::test_user_site_directory stdout ----
thread 'test::interpreter_config::test_user_site_directory' panicked at 'child exited unsuccessfully with exit code: 70', ~\.cargo\registry\src\github.com-1ecc6299db9ec823\rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:587:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```

Full test output
File paths redacted for readability.

```console
$ git describe --tags
pyoxidizer/0.10.0-380-g2b4e6640
$ cd pyembed
$ cargo test
Finished test [unoptimized + debuginfo] target(s) in 10.33s
Running PyOxidizer\target\debug\deps\pyembed-f8b1257a78853c69.exe

running 41 tests
test config::tests::test_packed_resources_explicit_origin ... ok
test config::tests::test_packed_resources_implicit_origin ... ok
test python_resources::tests::multiple_resource_blobs ... ok
test python_resources::tests::test_memory_mapped_file_resources ... ok
test test::importer::importer_indexing ... FAILED
test test::importer::importer_construction_py ... FAILED
test test::importer::builtins_py ... FAILED
test test::importer::find_spec_missing ... FAILED
test test::importer::importer_iter_modules_py ... FAILED
test test::importer::importer_module_loading_py ... FAILED
test test::importer::importer_metadata_py ... FAILED
test test::importer::importer_module_py ... FAILED
test test::importer::importer_resource_reading_py ... FAILED
test test::importer::importer_resource_collector_py ... FAILED
test test::importer::importer_resource_scanning_py ... FAILED
test test::importer::importer_resources_py ... FAILED
test test::importer::no_resources ... FAILED
test test::interpreter_config::test_argv_default ... FAILED
test test::interpreter_config::test_argv_override ... FAILED
test test::interpreter_config::test_argv_respect_interpreter_config ... FAILED
test test::interpreter_config::test_argvb_utf8 ... FAILED
test test::interpreter_config::test_argv_utf8_isolated_configure_locale ... FAILED
test test::interpreter_config::test_argv_utf8_isolated ... FAILED
test test::interpreter_config::test_argv_utf8 ... FAILED
test test::interpreter_config::test_bytes_warning_raise ... FAILED
test test::interpreter_config::test_bytes_warning_warn ... FAILED
test test::interpreter_config::test_default_interpreter ... FAILED
test test::interpreter_config::test_inspect ... FAILED
test test::interpreter_config::test_dev_mode ... FAILED
test test::interpreter_config::test_interactive ... FAILED
test test::interpreter_config::test_isolated_interpreter ... ok
test test::interpreter_config::test_optimization_level_one ... FAILED
test test::interpreter_config::test_optimization_level_two ... FAILED
test test::interpreter_config::test_quiet ... FAILED
test test::interpreter_config::test_site_import ... FAILED
test test::interpreter_config::test_sys_paths_origin ... ok
test test::interpreter_config::test_tcl_library_origin ... ok
test test::interpreter_config::test_use_environment ... FAILED
test test::interpreter_config::test_user_site_directory ... FAILED
test test::interpreter_config::test_utf8_mode ... FAILED
test test::interpreter_config::test_write_bytecode ... FAILED

failures:

---- test::importer::importer_indexing stdout ----
thread 'test::importer::importer_indexing' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:129:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_construction_py stdout ----
thread 'test::importer::importer_construction_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:124:54
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::builtins_py stdout ----
thread 'test::importer::builtins_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:112:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::find_spec_missing stdout ----
thread 'test::importer::find_spec_missing' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:91:44
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_iter_modules_py stdout ----
thread 'test::importer::importer_iter_modules_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:135:54
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_module_loading_py stdout ----
thread 'test::importer::importer_module_loading_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:165:56
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_metadata_py stdout ----
thread 'test::importer::importer_metadata_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:141:50
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_module_py stdout ----
thread 'test::importer::importer_module_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:118:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_resource_reading_py stdout ----
thread 'test::importer::importer_resource_reading_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:171:58
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_resource_collector_py stdout ----
thread 'test::importer::importer_resource_collector_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:147:60
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_resource_scanning_py stdout ----
thread 'test::importer::importer_resource_scanning_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:159:59
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::importer_resources_py stdout ----
thread 'test::importer::importer_resources_py' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: during initializing Python core: _PyConfig_Copy: memory allocation failed', pyembed\src\test\importer.rs:153:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::importer::no_resources stdout ----
thread 'test::importer::no_resources' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\importer.rs:70:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_default stdout ----
thread 'test::interpreter_config::test_argv_default' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:123:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_override stdout ----
thread 'test::interpreter_config::test_argv_override' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:181:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_respect_interpreter_config stdout ----
thread 'test::interpreter_config::test_argv_respect_interpreter_config' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:154:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argvb_utf8 stdout ----
thread 'test::interpreter_config::test_argvb_utf8' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:204:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_utf8_isolated_configure_locale stdout ----
thread 'test::interpreter_config::test_argv_utf8_isolated_configure_locale' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: , pvalue: Some("module 'sys' has no attribute 'argv'"), ptraceback: None }', pyembed\src\test\interpreter_config.rs:344:44
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_utf8_isolated stdout ----
thread 'test::interpreter_config::test_argv_utf8_isolated' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr { ptype: , pvalue: Some("module 'sys' has no attribute 'argv'"), ptraceback: None }', pyembed\src\test\interpreter_config.rs:281:44
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_argv_utf8 stdout ----
thread 'test::interpreter_config::test_argv_utf8' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:236:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_bytes_warning_raise stdout ----
thread 'test::interpreter_config::test_bytes_warning_raise' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:468:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_bytes_warning_warn stdout ----
thread 'test::interpreter_config::test_bytes_warning_warn' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:451:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_default_interpreter stdout ----
thread 'test::interpreter_config::test_default_interpreter' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:42:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_inspect stdout ----
thread 'test::interpreter_config::test_inspect' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:519:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_dev_mode stdout ----
thread 'test::interpreter_config::test_dev_mode' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:400:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_interactive stdout ----
thread 'test::interpreter_config::test_interactive' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:536:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_optimization_level_one stdout ----
thread 'test::interpreter_config::test_optimization_level_one' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:485:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_optimization_level_two stdout ----
thread 'test::interpreter_config::test_optimization_level_two' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:502:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_quiet stdout ----
thread 'test::interpreter_config::test_quiet' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:553:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_site_import stdout ----
thread 'test::interpreter_config::test_site_import' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:570:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_use_environment stdout ----
thread 'test::interpreter_config::test_use_environment' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:417:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_user_site_directory stdout ----
thread 'test::interpreter_config::test_user_site_directory' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:587:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_utf8_mode stdout ----
thread 'test::interpreter_config::test_utf8_mode' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:434:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test::interpreter_config::test_write_bytecode stdout ----
thread 'test::interpreter_config::test_write_bytecode' panicked at 'child exited unsuccessfully with exit code: 70', rusty-fork-0.3.0\src\fork_test.rs:135:9

running 1 test
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Dynamic("during initializing Python core: _PyConfig_Copy: memory allocation failed")', pyembed\src\test\interpreter_config.rs:604:61
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

failures:
test::importer::builtins_py
test::importer::find_spec_missing
test::importer::importer_construction_py
test::importer::importer_indexing
test::importer::importer_iter_modules_py
test::importer::importer_metadata_py
test::importer::importer_module_loading_py
test::importer::importer_module_py
test::importer::importer_resource_collector_py
test::importer::importer_resource_reading_py
test::importer::importer_resource_scanning_py
test::importer::importer_resources_py
test::importer::no_resources
test::interpreter_config::test_argv_default
test::interpreter_config::test_argv_override
test::interpreter_config::test_argv_respect_interpreter_config
test::interpreter_config::test_argv_utf8
test::interpreter_config::test_argv_utf8_isolated
test::interpreter_config::test_argv_utf8_isolated_configure_locale
test::interpreter_config::test_argvb_utf8
test::interpreter_config::test_bytes_warning_raise
test::interpreter_config::test_bytes_warning_warn
test::interpreter_config::test_default_interpreter
test::interpreter_config::test_dev_mode
test::interpreter_config::test_inspect
test::interpreter_config::test_interactive
test::interpreter_config::test_optimization_level_one
test::interpreter_config::test_optimization_level_two
test::interpreter_config::test_quiet
test::interpreter_config::test_site_import
test::interpreter_config::test_use_environment
test::interpreter_config::test_user_site_directory
test::interpreter_config::test_utf8_mode
test::interpreter_config::test_write_bytecode

test result: FAILED. 7 passed; 34 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--lib'
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in pyembed/src/test/interpreter_config.rs and pyembed/src/test/importer.rs, then run the pyembed cargo tests on Windows to reproduce the rusty-fork subprocess failures. Investigate the reported Python core _PyConfig_Copy memory-allocation error; done means the affected tests no longer exit with code 70 and the full 41-test run passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.