multiprocessing.Event.set() can deadlock when called from a signal handler or trace function

Bug report

Bug description:

multiprocessing.Event.set() will acquire a lock when setting the internal flag. multiprocessing.Event.is_set() will acquire the same lock when checking the flag. Thus if a signal handler calls .set() when .is_set() is running on the same process, there will be a deadlock.

multiprocessing.Event uses a regular non-reentrantlock lock. This should be changed to a reentrant lock. Please see the pull request.

Thanks for all the work on the Python programming language. I appreciate all your efforts highly.

Kind regards.

Example program below that (sometimes) deadlocks. On my machine I typically need to run it less than 10 times before a deadlock occurs. Also included in the code block is a sample stacktrace.

import faulthandler
import multiprocessing
import os
import signal


def run_buggy():
    shutdown_event = multiprocessing.Event()

    def sigterm_handler(_signo, _stack_frame):
        try:
            print(f'sigterm_handler running')
            shutdown_event.set()
        finally:
            print(f'sigterm_handler done')

    signal.signal(signal.SIGTERM, sigterm_handler)
    signal.signal(signal.SIGINT, sigterm_handler)
    faulthandler.register(signal.SIGUSR1)

    print(f'Running process with PID {os.getpid()}')
    print(f'Dump the stack by executing:')
    print(f'kill -SIGUSR1 {os.getpid()}')
    print(f'Try to kill this process with CTRL-C or kill {os.getpid()}')
    while not shutdown_event.is_set():
        pass

if __name__ == '__main__':
    run_buggy()
# Sample stacktrace:
# Current thread 0x00000001ed74cf40 (most recent call first):
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 95 in __enter__
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 237 in __enter__
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 342 in set
#   File "/Users/ire/code/cpython/./bug.py", line 13 in sigterm_handler
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 95 in __enter__
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 237 in __enter__
#   File "/Users/ire/code/cpython/Lib/multiprocessing/synchronize.py", line 335 in is_set
#   File "/Users/ire/code/cpython/./bug.py", line 25 in run_buggy
#   File "/Users/ire/code/cpython/./bug.py", line 29 in <module>

CPython versions tested on:

3.11, 3.12, CPython main branch

Operating systems tested on:

Linux, macOS

Linked PRs

• gh-126437

Start in Lib/multiprocessing/synchronize.py at the is_set() and set() paths referenced by the stack trace, then reproduce the issue with the example program and signal handler. Review linked PR gh-126437 and its tests; done means the Event no longer deadlocks when set() is called from the handler or trace function.