python / python/cpython

test_interpreters Failing Intermittently Under Py_DISABLE_GIL

Offen
#118,731 4 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen

Dieses Issue hat noch niemand übernommen.

3.13 topic-free-threading topic-subinterpreters type-bug
Vorherrschende Sprache
Python
Sterne
77.2k
Forks
36k
PR-Kennzahlen ausstehend

Beschreibung

Bug report

Bug description:

On one of my recent PRs, gh-118157, I was getting consistent failures on CI in test_interpreters. Different tests would fail, but the error was always the same:

(expand) 
0:08:24 load avg: 4.18 Re-running 1 failed tests in verbose mode in subprocesses
0:08:24 load avg: 4.18 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)
0:08:26 load avg: 3.93 [1/1/1] test_interpreters failed (6 failures)
Re-running test_interpreters in verbose mode (matching: test_display_preserved_exception, test_gh_109793, test_sys_path_0, test_sys_path_0, test_sys_path_0, test_sys_path_0)
test_display_preserved_exception (test.test_interpreters.test_api.TestInterpreterExec.test_display_preserved_exception) ... FAIL
test_gh_109793 (test.test_interpreters.test_lifecycle.FinalizationTests.test_gh_109793) ... FAIL
test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) ... 
### start subtest debug ###
# python3 script.py failed:
--- stdout ---
{
    "main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
    "sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000119aed0>
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x00005639aed8ea10)

------
### end subtest debug ###
  test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 script.py] ... FAIL
### start subtest debug ###
# python3 -m script failed:
--- stdout ---
{
    "main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
    "sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000117c0d0>
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x0000564591b58a10)

------
### end subtest debug ###
  test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m script] ... FAIL
### start subtest debug ###
# python3 -m pkg failed:
--- stdout ---
{
    "main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
    "sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000118c0d0>
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x000055aaa11eaa10)

------
### end subtest debug ###
  test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg] ... FAIL
### start subtest debug ###
# python3 -m pkg.script failed:
--- stdout ---
{
    "main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4",
    "sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4"
}
------
--- stderr ---
Exception ignored in: <function Interpreter.__del__ at 0x2000118c0d0>
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x00005637ca7cfa10)

------
### end subtest debug ###
  test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg.script] ... FAIL

======================================================================
FAIL: test_display_preserved_exception (test.test_interpreters.test_api.TestInterpreterExec.test_display_preserved_exception)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/__init__.py", line 2603, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_api.py", line 764, in test_display_preserved_exception
    self.assertEqual(stderr, dedent(f"""\
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
        Traceback (most recent call last):
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<18 lines>...
        RuntimeError: uh-oh!
        ^^^^^^^^^^^^^^^^^^^^
        """))
        ^^^^^
AssertionError: 'Trac[778 chars]oh!\nException ignored in: <function Interpret[519 chars]\n\n' != 'Trac[778 chars]oh!\n'
  Traceback (most recent call last):
    File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/script.py", line 9, in <module>
      interp.exec(script)
      ~~~~~~~~~~~^^^^^^^^
    File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 227, in exec
      raise ExecutionFailed(excinfo)
  test.support.interpreters.ExecutionFailed: RuntimeError: uh-oh!
  
  Uncaught in the interpreter:
  
  Traceback (most recent call last):
    File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/script.py", line 6, in script
      spam.eggs()
      ~~~~~~~~~^^
    File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/spam.py", line 6, in eggs
      ham()
      ~~~^^
    File "/tmp/test_python_tk7ha1xh/tmpkqxiakqz/spam.py", line 3, in ham
      raise RuntimeError('uh-oh!')
  RuntimeError: uh-oh!
- Exception ignored in: <function Interpreter.__del__ at 0x2000119aed0>
- Traceback (most recent call last):
-   File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
-   File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
- TypeError: catching classes that do not inherit from BaseException is not allowed
- Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
- Python runtime state: finalizing (tstate=0x000055b06f0a1a10)
- 


======================================================================
FAIL: test_gh_10[979](https://github.com/python/cpython/actions/runs/8978599353/job/24660038142#step:19:980)3 (test.test_interpreters.test_lifecycle.FinalizationTests.test_gh_109793)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 185, in test_gh_109793
    self.assertEqual(proc.returncode, 1)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 1

======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 script.py]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
    yield ctx
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
    out = self.run_python(argv, cwd=cwd)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
    self.assertEqual(proc.returncode, 0)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0

======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m script]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
    yield ctx
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
    out = self.run_python(argv, cwd=cwd)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
    self.assertEqual(proc.returncode, 0)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0

======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
    yield ctx
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
    out = self.run_python(argv, cwd=cwd)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
    self.assertEqual(proc.returncode, 0)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0

======================================================================
FAIL: test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) [python3 -m pkg.script]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 29, in subTest
    yield ctx
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 157, in test_sys_path_0
    out = self.run_python(argv, cwd=cwd)
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_interpreters/test_lifecycle.py", line 114, in run_python
    self.assertEqual(proc.returncode, 0)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: -6 != 0

----------------------------------------------------------------------
Ran 3 tests in 1.720s

FAILED (failures=6)
test test_interpreters failed
1 test failed again:
    test_interpreters

The error looks like this:

Exception ignored in: <function Interpreter.__del__ at 0x2000117c0d0>
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 157, in __del__
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/interpreters/__init__.py", line 173, in _decref
TypeError: catching classes that do not inherit from BaseException is not allowed
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x0000564591b58a10)

I'm fairly sure that means one of the 3 calls in _interpreters._decref() is raising a bogus exception.

It isn't clear to me how gh-118157 would have caused this to happen. As a very temporary solution, I've skipped test_interpreters on free-threading builds.

CC @colesbury

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Rechercherichtung

Beginne mit Lib/test/support/interpreters/init.py, insbesondere mit Interpreter.del und _decref(), und sieh dir anschließend test/test_interpreters/test_api.py und test/test_interpreters/test_lifecycle.py an. Führe test_interpreters mit aktiviertem Py_DISABLE_GIL aus und untersuche die gemeldete Ausnahme sowie die Fehler bei den verbleibenden Subinterpretern. Als erledigt gilt die Aufgabe, wenn die intermittierenden Fehler ohne den temporären Skip behoben sind.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
backend, testing
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.