python / python/cpython

Free-threading: `signal.pthread_kill` deadlocks child threads

Open
#150,873 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules topic-free-threading type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Context

The Dask threaded scheduler (e.g. dask.array or dask.dataframe without dask.distributed) blocks the main thread when the user calls compute() and internally spawns a concurrent.futures.ThreadPoolExecutor to schedule tasks. If a user hits Ctrl+C, they expect the program to terminate soon (at most waiting for the tasks that are halfway through execution to terminate).

Bug report

This sequence frequently deadlocks on Linux x64 free-threaded:

  1. main thread calls threading.Thread.start
  2. another thread calls signal.pthread_kill(main_thread, signal.SIGINT), which lands after start() has returned but before the thread's target function has started executing
  3. main thread handles the resulting KeyboardInterrupt
  4. main thread calls .join() on the child thread started at point 1.
  5. frequently, .join() deadlocks. A second SIGINT (e.g. from Ctrl+C) unblocks it.

Reproducer

import signal
import threading

USE_BARRIER = False
NTHREADS = 20

def demo():
    main_thread = threading.get_ident()
    barrier = threading.Barrier(NTHREADS) if USE_BARRIER else None
    thread_can_die = threading.Event()

    def f() -> None:
        if barrier:
            barrier.wait()
        thread_can_die.wait()

    def interrupt() -> None:
        if barrier:
            barrier.wait()
        signal.pthread_kill(main_thread, signal.SIGINT)

    threads = []
    for _ in range(NTHREADS - 1):
        t = threading.Thread(target=f)
        t.start()
        threads.append(t)
    try:
        t = threading.Thread(target=interrupt)
        t.start()
        threads.append(t)
        for t in threads:
            t.join()
    except KeyboardInterrupt:
        pass
    else:
        assert False, "Expected KeyboardInterrupt"

    thread_can_die.set()
    for t in threads:
        t.join()


if __name__ == "__main__":
    i = 0
    while True:
        print(i, flush=True)
        demo()
        i += 1

On these setups, the above program hangs within a handful of cycles:

  • Linux x64 3.14.5t
  • Linux x64 git tip #7a468a101268d2b13105f94ae027df8b502d0c87 (built with Tools/pixi-packages/freethreading)

Cannot reproduce on

  • Linux x64 3.14.5

Setting USE_BARRIER=True, which ensures that all child threads are already fully started when the SIGINT lands on the main thread, makes the issue disappear.

CPython versions tested on:

CPython main branch, 3.14

Operating systems tested on:

Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied reproducer on Linux x64 free-threaded CPython, then trace the interaction between threading.Thread.start(), threading.Thread.join(), and signal.pthread_kill(). Done means the reproducer no longer deadlocks when SIGINT arrives before a child thread's target begins, while the expected KeyboardInterrupt behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.