astral-sh / astral-sh/python-build-standalone

cpython ./configure's -pthread detection is misbehaving on some variants

Open
#599 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.4k
Forks
314
Avg merge
1d 9h
Merged PRs (30d)
27

Description

I am getting CI failures on #592 on some variants because of missing pthread symbols because of a missing `-pthread` on the compiler command line. Compare, for instance, [x86_64_v2-unknown-linux-gnu / 3.9 / debug](https://github.com/astral-sh/python-build-standalone/actions/runs/14718002189/job/41306023985?pr=592) which fails with

```
2025-04-28T21:36:18.8638916Z cpython-3.9> clang -L/tools/deps/lib -Wl,--exclude-libs,ALL -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o -L. -lpython3.9d -lpthread -ldl -lutil -lm -lbz2 -lffi -ldl -lm -lncursesw -lpanelw -lncursesw -ldb -lmpdec -lexpat -lcrypto -l:libatomic.a -llzma -lrt -lsqlite3 -lssl -lcrypto -l:libatomic.a -ltcl8.6 -ltk8.6 -lX11 -lxcb -lXau -luuid -lm -lm -lexpat -ledit -lncursesw -lz -lm
2025-04-28T21:36:19.1392376Z cpython-3.9> /tools/host/bin/ld: /tools/deps/lib/libtcl8.6.a(tclUnixNotfy.o): in function `Tcl_InitNotifier
2025-04-28T21:36:19.1394260Z cpython-3.9> ':
2025-04-28T21:36:19.1395858Z cpython-3.9> tclUnixNotfy.c:(.text+0x7d): undefined reference to `pthread_atfork'
2025-04-28T21:36:19.1910841Z cpython-3.9> clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

versus [x86_64-unknown-linux-gnu / 3.9 / debug](https://github.com/astral-sh/python-build-standalone/actions/runs/14718002189/job/41306015334?pr=592) which succeeds with the corresponding build line being

```
2025-04-28T21:37:46.4684015Z cpython-3.9> clang -pthread -L/tools/deps/lib -Wl,--exclude-libs,ALL -Xlinker -export-dynamic -o Programs/_testembed Programs/_testembed.o -L. -lpython3.9d -lpthread -ldl -lutil -lm -lbz2 -lffi -ldl -lm -lncursesw -lpanelw -lncursesw -ldb -lmpdec -lexpat -lcrypto -l:libatomic.a -llzma -lrt -lsqlite3 -lssl -lcrypto -l:libatomic.a -ltcl8.6 -ltk8.6 -lX11 -lxcb -lXau -luuid -lm -lm -lexpat -ledit -lncursesw -lz -lm
```

If you dig a bit earlier in the build logs, the x86_64_v2 (unsuccessful) build does

```
2025-04-28T21:35:49.0615195Z cpython-3.9> checking whether pthreads are available without options...
2025-04-28T21:35:49.0616630Z cpython-3.9> no
2025-04-28T21:35:49.0618854Z cpython-3.9> checking whether clang accepts -Kpthread...
2025-04-28T21:35:49.0619625Z cpython-3.9> no
2025-04-28T21:35:49.0622574Z cpython-3.9> checking whether clang accepts -Kthread... no
2025-04-28T21:35:49.0622969Z cpython-3.9> checking whether clang accepts -pthread...
2025-04-28T21:35:49.0623276Z cpython-3.9> no
2025-04-28T21:35:49.0623591Z cpython-3.9> checking whether clang++ also accepts flags for thread support...
2025-04-28T21:35:49.0623927Z cpython-3.9> no
```

while the x86_64 (successful) build does

```
2025-04-28T21:37:20.2180518Z cpython-3.9> checking whether pthreads are available without options...
2025-04-28T21:37:20.2551345Z cpython-3.9> no
2025-04-28T21:37:20.2552261Z cpython-3.9> checking whether clang accepts -Kpthread...
2025-04-28T21:37:20.2676725Z cpython-3.9> no
2025-04-28T21:37:20.2677100Z cpython-3.9> checking whether clang accepts -Kthread...
2025-04-28T21:37:20.2800777Z cpython-3.9> no
2025-04-28T21:37:20.2804876Z cpython-3.9> checking whether clang accepts -pthread...
2025-04-28T21:37:20.3187186Z cpython-3.9> yes
2025-04-28T21:37:20.3187607Z cpython-3.9> checking whether clang++ also accepts flags for thread support...
2025-04-28T21:37:20.3793651Z cpython-3.9> yes
```

This does not appear to have been introduced by my PR; the same ./configure discrepancy happens on HEAD (compare [this run](https://github.com/astral-sh/python-build-standalone/actions/runs/14361390782/job/40264100181) and [this run](https://github.com/astral-sh/python-build-standalone/actions/runs/14361390782/job/40264036290), for instance), though the fact that ./configure decides not to pass `-pthread` does not fail the build on HEAD. I do not know if it has any unwanted side effects.

Almost certainly what is going on is that the test program is failing to run for some other reason, and ./configure is getting confused.

We need to resolve this as it blocks #592, and I'm a little bit uncomfortable with the fact that this discrepancy is happening and I'd like to think a bit about noticing and detecting discrepancies like this in the general case.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.