tests testoutput.py::OutputJSONsuite can be flaky on high-core-count systems
Open
Nobody has claimed this yet.
bug
parallel checking
topic-developer
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Now that Debian is testing mypy with native parsing (via ast-serialize) we resumed running the entire test suite.
With mypy 2.2.0 we see random failures of testoutput.py::OutputJSONsuite::outputjson.test::* on high-core count systems.
Mostly due to ResourceWarning: subprocess 21441 is still running being turned into pytest.PytestUnraisableExceptionWarning: Exception ignored while calling deallocator <function Popen.__del__ at 0x7f30bbe13ed0>: None
But also other errors: mypy.errors.CompileError: {"file": "tmp/main", "line": 2, "column": 0, "end_line": 2, "end_column": 5, "message": "\"break\" outside loop", "hint": null, "code": null, "severity": "error"} and OSError: Cannot connect to build worker(s): 0
Details
Here we had test failures under Python 3.13 and 3.14 https://ci.debian.net/packages/m/mypy/testing/ppc64el/74734444/
2227s =================================== FAILURES ===================================
2227s ____________________________ testOutputJsonParallel ____________________________
2227s [gw10] linux -- Python 3.13.15 /usr/bin/python3.13
2227s data: /tmp/autopkgtest.EyGnnA/build.z2o/src/test-data/unit/outputjson.test:46:
2227s /usr/lib/python3.13/subprocess.py:1140: in __del__
2227s _warn("subprocess %s is still running" % self.pid,
2227s E ResourceWarning: subprocess 6479 is still running
2227s
2227s The above exception was the direct cause of the following exception:
2227s /usr/lib/python3/dist-packages/pluggy/_hooks.py:512: in __call__
2227s return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
2227s ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2227s /usr/lib/python3/dist-packages/pluggy/_manager.py:120: in _hookexec
2227s return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
2227s ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/logging.py:865: in pytest_runtest_call
2227s yield
2227s /usr/lib/python3/dist-packages/_pytest/capture.py:900: in pytest_runtest_call
2227s return (yield)
2227s ^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/skipping.py:268: in pytest_runtest_call
2227s return (yield)
2227s ^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:183: in pytest_runtest_call
2227s collect_unraisable(item.config)
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:79: in collect_unraisable
2227s raise errors[0]
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:67: in collect_unraisable
2227s warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
2227s E pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Popen.__del__ at 0x3fff82b01300>
2227s E Enable tracemalloc to get traceback where the object was allocated.
2227s E See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
2227s ______________________ testOutputJsonSyntaxErrorParallel _______________________
2227s [gw10] linux -- Python 3.13.15 /usr/bin/python3.13
2227s data: /tmp/autopkgtest.EyGnnA/build.z2o/src/test-data/unit/outputjson.test:62:
2227s mypy/main.py:244: in run_build
2227s ???
2227s mypy/build.py:422: in build
2227s result = build_inner(
2227s mypy/build.py:537: in build_inner
2227s graph = dispatch(sources, manager, stdout, connect_threads)
2227s mypy/build.py:4146: in dispatch
2227s process_graph(graph, manager)
2227s mypy/build.py:4614: in process_graph
2227s done, still_working, results = manager.wait_for_done(graph)
2227s mypy/build.py:1480: in wait_for_done
2227s return self.wait_for_done_workers(graph)
2227s mypy/build.py:1506: in wait_for_done_workers
2227s raise data.blocker
2227s E mypy.errors.CompileError: {"file": "tmp/main", "line": 2, "column": 0, "end_line": 2, "end_column": 5, "message": "\"break\" outside loop", "hint": null, "code": null, "severity": "error"}
2227s
2227s During handling of the above exception, another exception occurred:
2227s /usr/lib/python3.13/subprocess.py:1140: in __del__
2227s _warn("subprocess %s is still running" % self.pid,
2227s E ResourceWarning: subprocess 6518 is still running
2227s
2227s The above exception was the direct cause of the following exception:
2227s /usr/lib/python3/dist-packages/pluggy/_hooks.py:512: in __call__
2227s return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
2227s ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2227s /usr/lib/python3/dist-packages/pluggy/_manager.py:120: in _hookexec
2227s return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
2227s ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/logging.py:865: in pytest_runtest_call
2227s yield
2227s /usr/lib/python3/dist-packages/_pytest/capture.py:900: in pytest_runtest_call
2227s return (yield)
2227s ^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/skipping.py:268: in pytest_runtest_call
2227s return (yield)
2227s ^^^^^
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:183: in pytest_runtest_call
2227s collect_unraisable(item.config)
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:79: in collect_unraisable
2227s raise errors[0]
2227s /usr/lib/python3/dist-packages/_pytest/unraisableexception.py:67: in collect_unraisable
2227s warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
2227s E pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Popen.__del__ at 0x3fff82b01300>
2227s E Enable tracemalloc to get traceback where the object was allocated.
2227s E See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
More examples:
- ppc64el just with Python 3.13: https://ci.debian.net/packages/m/mypy/testing/ppc64el/74815096/
- amd64 failures with Python 3.13 & 3.14 (48 workers) https://ci.debian.net/packages/m/mypy/testing/amd64/74733699/ ; but passed just fine on a retry: https://ci.debian.net/packages/m/mypy/testing/amd64/74785836/
- arm64 different errors, same test cases: https://ci.debian.net/packages/m/mypy/testing/arm64/74734176 (linked to from the test error summary at the top)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with testoutput.py::OutputJSONsuite and the cases in test-data/unit/outputjson.test at lines 46 and 62. Reproduce the failures with parallel workers on a high-core-count system and Python 3.13 or 3.14, then trace the reported subprocess and build-worker errors. Done means these tests pass reliably without ResourceWarning, CompileError, or worker-connection failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100