llvm / llvm/llvm-project

[lldb] TestRunLocker.py is flakey in pre-commit CI

Open
#193,787 3 comments 0 reactions 0 assignees View on GitHub
lldb test-suite
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

```
Script:
--
/usr/bin/python3 /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --env LLVM_INCLUDE_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/include --env LLVM_TOOLS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --libcxx-include-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/c++/v1 --libcxx-include-target-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/x86_64-unknown-linux-gnu/c++/v1 --libcxx-library-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib/x86_64-unknown-linux-gnu --triple x86_64-unknown-linux-gnu --build-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build.noindex --lldb-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/lldb --compiler /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/clang --dsymutil /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --lldb-obj-root /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb --lldb-libs-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --cmake-build-type Release /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/python_api/run_locker -p TestRunLocker.py
--
Exit Code: 1

Command Output (stdout):
--
lldb version 23.0.0git (https://github.com/llvm/llvm-project revision d90e3be11859d467437d0edc74782ed923e61513)
clang revision d90e3be11859d467437d0edc74782ed923e61513
llvm revision d90e3be11859d467437d0edc74782ed923e61513
"can't evaluate expressions when the process is running."
Skipping the following test categories: msvcstl, dsym, pdb, gmodules, debugserver, objc

--
Command Output (stderr):
--
Traceback (most recent call last):
File "", line 1, in
AttributeError: 'NoneType' object has no attribute 'EvaluateExpression'
FAIL: LLDB (/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang-x86_64) :: test_run_locker (TestRunLocker.TestRunLocker.test_run_locker)
Log Files:
- /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build.noindex/python_api/run_locker/TestRunLocker/Failure_test_run_locker.log
PASS: LLDB (/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang-x86_64) :: test_run_locker_stop_at_entry (TestRunLocker.TestRunLocker.test_run_locker_stop_at_entry)
======================================================================
FAIL: test_run_locker (TestRunLocker.TestRunLocker.test_run_locker)
Test that the run locker is set correctly when we launch
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/packages/Python/lldbsuite/test/decorators.py", line 160, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/python_api/run_locker/TestRunLocker.py", line 23, in test_run_locker
self.runlocker_test(False)
File "/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/python_api/run_locker/TestRunLocker.py", line 110, in runlocker_test
self.assertIn(
AssertionError: "can't evaluate expressions when the process is running" not found in ''
Config=x86_64-/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang
----------------------------------------------------------------------
Ran 2 tests in 0.553s

FAILED (failures=1)

--
```

Example: https://github.com/llvm/llvm-project/pull/193118

Contributor guide

Open the contributing guide

Research direction

Start with lldb/test/API/python_api/run_locker/TestRunLocker.py, especially runlocker_test and the assertion around line 110. Run the test with -p TestRunLocker.py using the reported LLDB test command, then compare repeated runs with the failure log. Done means the test reliably handles the launch-time run-locker behavior and passes in pre-commit CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.