python / python/cpython

Data race in `faulthandler.enable()` and `faulthandler.disable()` with free-threading

Open
#151,363 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

faulthandler.enable() and faulthandler.disable() read and write the global fatal_error state with no synchronization. enable() checks the enabled guard and then sets it,

https://github.com/python/cpython/blob/b18168cb32d545ed976b760983478cbd5dde5bdf/Modules/faulthandler.c#L536-L541

while disable() reads the same guard and tears the state back down, including Py_CLEAR(fatal_error.file),

https://github.com/python/cpython/blob/b18168cb32d545ed976b760983478cbd5dde5bdf/Modules/faulthandler.c#L641-L658

so two threads calling enable() and disable() concurrently race on the enabled flag, on the installed signal handlers, and on the fatal_error.file reference.

Reproducer:

import faulthandler, os
from threading import Thread, Event

def owner():
    f = open(os.devnull, 'w')
    for _ in range(200000):
        faulthandler.enable(file=f, all_threads=False)
        f.write('x')
        f.flush()
    f.close()

def toggler():
    for _ in range(200000):
        faulthandler.disable()

threads  = [Thread(target=owner)   for _ in range(4)]
threads += [Thread(target=toggler) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()

With TSAN build, the owner thread's own f.write()/enable() raises ValueError: I/O operation on uninitialized object because a concurrent disable() dropped the last reference to f and finalized it.

TSAN Report:

WARNING: ThreadSanitizer: data race (pid=1671402)
  Write of size 4 at 0x555555e19478 by thread T5:
    #0 faulthandler_disable /cpython/./Modules/faulthandler.c:644:29 (python3.16t+0x57830e) 
    #1 faulthandler_disable_py_impl /cpython/./Modules/faulthandler.c:674:5  
    #2 faulthandler_disable_py /cpython/./Modules/clinic/faulthandler.c.h:299:12 
    #3 cfunction_vectorcall_NOARGS /cpython/Objects/methodobject.c:508:24  
    #4 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:144:11  
    #5 PyObject_Vectorcall /cpython/Objects/call.c:327:12 
...

  Previous read of size 4 at 0x555555e19478 by thread T4:
    #0 faulthandler_enable /cpython/./Modules/faulthandler.c:538:21  
    #1 faulthandler_py_enable_impl /cpython/./Modules/faulthandler.c:633:9 
    #2 faulthandler_py_enable /cpython/./Modules/clinic/faulthandler.c.h:278:20 
    #3 cfunction_vectorcall_FASTCALL_KEYWORDS /cpython/Objects/methodobject.c:465:24  
    #4 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:144:11  
    #5 PyObject_Vectorcall /cpython/Objects/call.c:327:12 
...

  Location is global '_PyRuntime' of size 405824 at 0x555555e16c80 

SUMMARY: ThreadSanitizer: data race /cpython/./Modules/faulthandler.c:644:29 in faulthandler_disable
==================
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Modules/faulthandler.c at the enable() lines 536-541 and disable() lines 641-658, then run the provided reproducer with a ThreadSanitizer build. Trace the shared enabled state, signal handlers, and fatal_error.file ownership; done means concurrent enable()/disable() calls no longer produce the reported race or invalidate the caller's file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
46/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.