Prevent deadlocks in `ctypes.util.dllist()` on Linux (and maybe other Unices)

On Linux, ctypes.util.dllist() uses dl_iterate_phdr, which due to the way it works is prone to deadlocks when used from Python. It would be good if Python guaranteed that this form of deadlock won't happen. Update: See reproducer below, in the comments.

This is relevant to threadpoolctl (183 million PyPI downloads last month), which has started to use ctypes.util.dlllist() in its latest release, and in an ideal world would only rely on it once Python 3.14 is the oldest supported release.

Background: A real deadlock bug involving dl_iterate_phdr I encountered in the real world

In older releases, threadpoolctl would use dl_iterate_phdr itself (using ctypes). Importantly, this C API uses callbacks, and so threadpoolctl had the callbacks call back into Python.

Meanwhile, NumPy on Linux also calls dl_iterate_phdr indirectly (via backtrace API on Linux, some sort of temporary array optimization). NumPy holds the GIL while doing so (I will file an issue about that too).

threadpoolctl is very often used together with NumPy, so now that threading is becoming more commonly used, this interaction is a likely scenario. And presumably other libraries use dl_iterate_phdr, either directly, or indirectly as NumPy does.

So here's the scenario:

• Thread A is calling dl_iterate_phdr via threadpoolctl.
• Thread B is calling dl_iterate_phdr via NumPy.

The following sequence of events can happen:

1. Thread A calls dl_iterate_phdr. This acquires an internal lock inside the dl system; we'll call this DL-LOCK.
2. Thread A calls into Python, and the GIL gets released inside that callback.
3. Thread B gets scheduled, it holds the GIL.
4. Thread B calls dl_iterate_phdr, which tries to acquire DL-LOCK... but it's held by thread A. Meanwhile, thread A is waiting for the GIL, which is held by thread B.

At this point the program is deadlocked, and no Python thread will ever run again.

ctypes.util.dllist() in CPython seems potentially vulnerable to this deadlock

Looking at the relevant code in CPython, the same deadlock could happen if PyUnicode_DecodeFSDefault or PyList_Append release the GIL.

The former at least ends up dealing with the whole codec lookup system, plus some specific filesystem encoding infrastructure, so what it does is pretty complex. It might release the GIL, or it might not. And it might potentially change to not be OK in future versions.

The solution

When using dl_iterate_phdr, dllist() should have as part of its contract that the GIL can't be released and reacquired while calling dl_iterate_phdr.

More concretely:

1. Not call into Python at all in the dl_iterate_phdr callback. Instead, accumulate the paths in pure C data structures, and then do the Python transformation once dl_iterate_phdr is done.
2. Since dl_iterate_phdr is acquiring an external non-Python lock, it should also be wrapped with Py_BEGIN_ALLOW_THREADS.

Comece em Modules/_ctypes/callproc.c, na implementação vinculada de dllist(), depois leia o contrato de dl_iterate_phdr e o reprodutor referenciado nos comentários. Garanta que o callback não chame Python enquanto a iteração de bibliotecas estiver ativa e verifique se o cenário de deadlock GIL/DL-LOCK relatado não é mais possível.