llvm / llvm/llvm-project

[lldb] TestDataFormatterSynth.py is flaky on lldb-aarch64-windows

Open
#219,692 2 comments 0 reactions 0 assignees View on GitHub
infrastructure lldb platform:windows test-suite
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

`lldb-api :: functionalities/data-formatter/data-formatter-synth/TestDataFormatterSynth.py` intermittently fails on the [`lldb-aarch64-windows`](https://lab.llvm.org/buildbot/#/builders/141) builder running on `linaro-armv8-windows-msvc-05`.

Failing builds:

- [22303](https://lab.llvm.org/buildbot/#/builders/141/builds/22303), revision [`f1537cf2b5b1`](https://github.com/llvm/llvm-project/commit/f1537cf2b5b1d210b2fd7f4dda7c99c07d73a257)
- [22306](https://lab.llvm.org/buildbot/#/builders/141/builds/22306), revision [`e94b49e4aaef`](https://github.com/llvm/llvm-project/commit/e94b49e4aaefea3c50995e5c6229bfa5846e2ed3)

Builds [22304](https://lab.llvm.org/buildbot/#/builders/141/builds/22304) and [22305](https://lab.llvm.org/buildbot/#/builders/141/builds/22305) were green between the two failures, and `TestDataFormatterSynth.py` passed in build 22305. There were no changes under `lldb/` between the two failing revisions.

At the same `e94b49e4aaef` revision as failing build 22306, the test explicitly passed on the Windows x86_64 [`lldb-x86_64-win` build 10314](https://lab.llvm.org/buildbot/#/builders/211/builds/10314), running on `as-builder-10`. This confirms a pass on one different Windows worker and configuration.

Both failures report the same traceback:

```text
FAIL: test_with_run_command_dwarf (TestDataFormatterSynth.SynthDataFormatterTestCase.test_with_run_command_dwarf)

Traceback (most recent call last):
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2187, in test_method
return attrvalue(self)
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\test\API\functionalities\data-formatter\data-formatter-synth\TestDataFormatterSynth.py", line 199, in test_with_run_command
self.expect("frame variable bag_bag", substrs=["x.z = 12"])
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 2867, in expect
self.runCmd(
File "C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1133, in runCmd
self.assertTrue(self.res.Succeeded(), msg + output)
AssertionError: False is not true : Command 'frame variable bag_bag' did not return successfully
Error output:
error: Command requires a process which is currently stopped.

Config=aarch64-C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\bin\clang.exe
```

The failing assertion immediately follows a synchronous `n` command:

```python
self.runCmd("n")
self.expect("frame variable bag_bag", substrs=["x.z = 12"])
```

The inferior has only an assignment and `return 0` after the initial breakpoint:

```cpp
plenty_of_stuff.bitfield = 0x11; // Set break point at this line.

bag_bag.x.z = 12;

return 0;
```

This suggests that the second `n` occasionally lets the process exit instead of leaving it stopped at the expected source location. The test command is synchronous, so by the time the following `frame variable` command runs, the inferior has either stopped or exited.

Possibly related: #201068 reports a similar observed failure mode on the same Windows on Arm builder, where the inferior intermittently exits after synchronous stepping and before the next command. It affects a different test, and the common downstream "process is not stopped" error does not establish a common root cause. I could not find an existing report for `TestDataFormatterSynth.py` itself.

I have not reproduced this locally or exhaustively checked every other LLDB builder.

Contributor guide

Open the contributing guide

Research direction

Start with lldb/test/API/functionalities/data-formatter/data-formatter-synth/TestDataFormatterSynth.py, especially test_with_run_command_dwarf around line 199, and run it on the lldb-aarch64-windows builder. Investigate the synchronous n command and whether the inferior exits before frame variable bag_bag runs, comparing the related behavior in issue #201068. Done means the intermittent failure is reproduced or explained and the test reliably completes with the expected stopped frame state.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.