TestTwoHitsOneActual.py is flakey on Windows on Arm
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Has been failing a lot on Linaro's bot (https://github.com/llvm/llvm-project/pulls?q=is%3Apr+TestTwoHitsOneActual+is%3Aclosed) and on our currently downstream equivalent of that.
```
FAIL: lldb-api :: functionalities/breakpoint/two_hits_one_actual/TestTwoHitsOneActual.py (396 of 2968)
******************** TEST 'lldb-api :: functionalities/breakpoint/two_hits_one_actual/TestTwoHitsOneActual.py' FAILED ********************
Script:
--
C:/ProgramData/scoop/apps/python/current/python.exe C:/buildbot/workspace/lldb-aarch64-windows/llvm-project/lldb\test\API\dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=C:/buildbot/workspace/lldb-aarch64-windows/build/./lib --env LLVM_INCLUDE_DIR=C:/buildbot/workspace/lldb-aarch64-windows/build/include --env LLVM_TOOLS_DIR=C:/buildbot/workspace/lldb-aarch64-windows/build/./bin --triple aarch64-pc-windows-msvc --build-dir C:/buildbot/workspace/lldb-aarch64-windows/build/lldb-test-build --lldb-module-cache-dir C:/buildbot/workspace/lldb-aarch64-windows/build/lldb-test-build/module-cache-lldb\lldb-api --clang-module-cache-dir C:/buildbot/workspace/lldb-aarch64-windows/build/lldb-test-build/module-cache-clang\lldb-api --executable C:/buildbot/workspace/lldb-aarch64-windows/build/./bin/lldb.exe --lldb-python-dir C:\buildbot\workspace\lldb-aarch64-windows\build\Lib\site-packages --compiler C:/buildbot/workspace/lldb-aarch64-windows/build/./bin/clang.exe --dsymutil C:/buildbot/workspace/lldb-aarch64-windows/build/./bin/dsymutil.exe --make C:/ProgramData/scoop/shims/make.exe --llvm-tools-dir C:/buildbot/workspace/lldb-aarch64-windows/build/./bin --lldb-obj-root C:/buildbot/workspace/lldb-aarch64-windows/build/tools/lldb --lldb-libs-dir C:/buildbot/workspace/lldb-aarch64-windows/build/./lib --cmake-build-type Release --skip-category=watchpoint --env LLDB_LAUNCH_FLAG_USE_PIPES=1 C:\buildbot\workspace\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\breakpoint\two_hits_one_actual -p TestTwoHitsOneActual.py --timeout 600
--
Exit Code: 1
Command Output (stdout):
--
Skipping the following test categories: watchpoint, libc++, libstdcxx, dwo, dsym, gmodules, debugserver, objc, fork, pexpect
--
Command Output (stderr):
--
FAIL: LLDB (C:\buildbot\workspace\lldb-aarch64-windows\build\bin\clang.exe-aarch64) :: test_two_hits_one_actual (TestTwoHitsOneActual.TestTwoHitsOneActual.test_two_hits_one_actual)
Log Files:
- C:\buildbot\workspace\lldb-aarch64-windows\build\lldb-test-build\functionalities\breakpoint\two_hits_one_actual\TestTwoHitsOneActual\Failure_test_two_hits_one_actual.log
======================================================================
FAIL: test_two_hits_one_actual (TestTwoHitsOneActual.TestTwoHitsOneActual.test_two_hits_one_actual)
There can be many tests in a test case - describe this test here.
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\buildbot\workspace\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\decorators.py", line 221, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\buildbot\workspace\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\breakpoint\two_hits_one_actual\TestTwoHitsOneActual.py", line 23, in test_two_hits_one_actual
self.sample_test()
File "C:\buildbot\workspace\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\breakpoint\two_hits_one_actual\TestTwoHitsOneActual.py", line 65, in sample_test
self.assertStopReason(
File "C:\buildbot\workspace\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 3104, in assertStopReason
self.fail(self._formatMessage(msg, error))
AssertionError: none (1) != breakpoint (3)
Config=aarch64-C:\buildbot\workspace\lldb-aarch64-windows\build\bin\clang.exe
----------------------------------------------------------------------
```
No idea what's happening but the test is very similar to https://github.com/llvm/llvm-project/issues/220942 so it could be the same problem.
Running individually I cannot get it to fail. If it's a timing issue, that would make some sense.
Contributor guide
Research direction
Start with lldb/test/API/functionalities/breakpoint/two_hits_one_actual/TestTwoHitsOneActual.py, especially sample_test and the failing assertStopReason call. Run the test on the Windows Arm configuration and compare individual runs with the bot or suite results; also review issue 220942 for the suspected related failure. Done means the test no longer flakes on the affected configuration and passes reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools, operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100