Potential thread unsafety in test_free_threading.test_monitoring

Bug report

Bug description:

It seems that there is some kind of race with test_free_threading.test_monitoring, specifically the sys.settrace tests:
https://github.com/python/cpython/blob/fba5dded6df3c2b1943557afef89a5cb418f65a2/Lib/test/test_free_threading/test_monitoring.py#L143-L166.

This was initially spotted in a buildbot failure here: https://buildbot.python.org/#/builders/1610/builds/1590. There have been a few behaviors noted:

• refleaks causing a test failure
• assertion failures
• presumed deadlocks or other contention causing the tests to run extremely slowly (timing out after 45 minutes)
• tests pass without issue 😬

The variety of failure modes makes me think there is some kind of thread safety issue lurking behind the failures...

After some local debugging (the remote debugger rocks!), I was able to reproduce the extremely long test run of test_freethreading, and narrow down the majority of the time being in the sys.settrace tests. My command line was:

./python -E  -m test --timeout=2700 -R 3:3 -u-cpu -j10 test_free_threading

On my x86 laptop on Ubuntu 24.04/WSL.

Reading over the test, it isn't clear to me what is meant to be tested. sys.settrace only affects the current thread according to the docs (indeed, the C code sets the trace function to the current threadstate), so currently it is only checking that the main thread executes a frame of some sort.

I didn't have time to find the root cause of the weird behavior above, so I'm filing this bug to note down my findings.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Start with Lib/test/test_free_threading/test_monitoring.py lines 143-166 and reproduce with ./python -E -m test --timeout=2700 -R 3:3 -u-cpu -j10 test_free_threading. Investigate the sys.settrace tests alongside the linked buildbot failure and determine the source of the race, contention, or leaks. Done means the failure mode is understood and the affected tests run reliably.