[lldb] TestDataFormatterSynth.py is flaky on lldb-aarch64-windows
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
`lldb-api :: functionalities/data-formatter/data-formatter-synth/TestDataFormatterSynth.py` intermittently fails on the [`lldb-aarch64-windows`](https://lab.llvm.org/buildbot/#/builders/141) builder running on `linaro-armv8-windows-msvc-05`.
Failing builds:
- [22303](https://lab.llvm.org/buildbot/#/builders/141/builds/22303), revision [`f1537cf2b5b1`](https://github.com/llvm/llvm-project/commit/f1537cf2b5b1d210b2fd7f4dda7c99c07d73a257)
- [22306](https://lab.llvm.org/buildbot/#/builders/141/builds/22306), revision [`e94b49e4aaef`](https://github.com/llvm/llvm-project/commit/e94b49e4aaefea3c50995e5c6229bfa5846e2ed3)
Builds [22304](https://lab.llvm.org/buildbot/#/builders/141/builds/22304) and [22305](https://lab.llvm.org/buildbot/#/builders/141/builds/22305) were green between the two failures, and `TestDataFormatterSynth.py` passed in build 22305. There were no changes under `lldb/` between the two failing revisions.
At the same `e94b49e4aaef` revision as failing build 22306, the test explicitly passed on the Windows x86_64 [`lldb-x86_64-win` build 10314](https://lab.llvm.org/buildbot/#/builders/211/builds/10314), running on `as-builder-10`. This confirms a pass on one different Windows worker and configuration.
Both failures report the same traceback:
```text
FAIL: test_with_run_command_dwarf (TestDataFormatterSynth.SynthDataFormatterTestCase.test_with_run_command_dwarf)
Traceback (most recent call last):
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2187, in test_method
return attrvalue(self)
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\data-formatter\data-formatter-synth\TestDataFormatterSynth.py", line 199, in test_with_run_command
self.expect("frame variable bag_bag", substrs=["x.z = 12"])
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2867, in expect
self.runCmd(
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1133, in runCmd
self.assertTrue(self.res.Succeeded(), msg + output)
AssertionError: False is not true : Command 'frame variable bag_bag' did not return successfully
Error output:
error: Command requires a process which is currently stopped.
Config=aarch64-C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\bin\clang.exe
```
The failing assertion immediately follows a synchronous `n` command:
```python
self.runCmd("n")
self.expect("frame variable bag_bag", substrs=["x.z = 12"])
```
The inferior has only an assignment and `return 0` after the initial breakpoint:
```cpp
plenty_of_stuff.bitfield = 0x11; // Set break point at this line.
bag_bag.x.z = 12;
return 0;
```
This suggests that the second `n` occasionally lets the process exit instead of leaving it stopped at the expected source location. The test command is synchronous, so by the time the following `frame variable` command runs, the inferior has either stopped or exited.
Possibly related: #201068 reports a similar observed failure mode on the same Windows on Arm builder, where the inferior intermittently exits after synchronous stepping and before the next command. It affects a different test, and the common downstream "process is not stopped" error does not establish a common root cause. I could not find an existing report for `TestDataFormatterSynth.py` itself.
I have not reproduced this locally or exhaustively checked every other LLDB builder.
Contributor guide
Research direction
Start with lldb/test/API/functionalities/data-formatter/data-formatter-synth/TestDataFormatterSynth.py, especially test_with_run_command_dwarf around line 199, and run it on the lldb-aarch64-windows builder. Investigate the synchronous n command and whether the inferior exits before frame variable bag_bag runs, comparing the related behavior in issue #201068. Done means the intermittent failure is reproduced or explained and the test reliably completes with the expected stopped frame state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- devtools, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100