TestFrameProviderCircularDependency.py is flakey in GitHub CI

Open

#212,761 opened on Jul 29, 2026 9 comments 0 reactions 1 assignee Claimed by @medismailben View on GitHub

infrastructure lldb test-suite

Dominant language: LLVM
Stars: 40.5k
Forks: 18.7k
PR merge metrics: PR metrics pending

Description

TestFrameProviderCircularDependency.py has been failing in CI recently. Some examples I found:

https://github.com/llvm/llvm-project/pull/212066 (from an older edit of the CI comment):
```
Script:
--
/usr/bin/python3 /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --env LLVM_INCLUDE_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/include --env LLVM_TOOLS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --libcxx-include-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/c++/v1 --libcxx-include-target-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/x86_64-unknown-linux-gnu/c++/v1 --libcxx-library-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib/x86_64-unknown-linux-gnu --triple x86_64-unknown-linux-gnu --build-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build --lldb-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build/module-cache-lldb/lldb-api --clang-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build/module-cache-clang/lldb-api --executable /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/lldb --lldb-python-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/local/lib/python3.12/dist-packages --compiler /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/clang --dsymutil /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --lldb-obj-root /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb --lldb-libs-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --cmake-build-type Release /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/functionalities/scripted_frame_provider/circular_dependency -p TestFrameProviderCircularDependency.py
--
Exit Code: 1

Command Output (stdout):
--
Skipping the following test categories: msvcstl, dsym, pdb, gmodules, debugserver, objc

--
Command Output (stderr):
--
FAIL: LLDB (/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang-x86_64) :: test_circular_dependency_evaluate_expression_in_get_frame (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase.test_circular_dependency_evaluate_expression_in_get_frame)
Log Files:
- /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build/functionalities/scripted_frame_provider/circular_dependency/TestFrameProviderCircularDependency/Failure_test_circular_dependency_evaluate_expression_in_get_frame.log
PASS: LLDB (/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang-x86_64) :: test_circular_dependency_handle_command_in_init (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase.test_circular_dependency_handle_command_in_init)
PASS: LLDB (/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang-x86_64) :: test_circular_dependency_with_function_replacement (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase.test_circular_dependency_with_function_replacement)
======================================================================
FAIL: test_circular_dependency_evaluate_expression_in_get_frame (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase.test_circular_dependency_evaluate_expression_in_get_frame)
Test that calling EvaluateExpression in get_frame_at_index doesn't
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/functionalities/scripted_frame_provider/circular_dependency/TestFrameProviderCircularDependency.py", line 204, in test_circular_dependency_evaluate_expression_in_get_frame
self.assertIsNotNone(
AssertionError: unexpectedly None : Frame 1 should have function name
Config=x86_64-/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang
----------------------------------------------------------------------
Ran 3 tests in 36.197s

FAILED (failures=1)
```
https://github.com/llvm/llvm-project/pull/212015:
```
Script:
--
/usr/bin/python3 /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --env LLVM_INCLUDE_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/include --env LLVM_TOOLS_DIR=/home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --libcxx-include-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/c++/v1 --libcxx-include-target-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/include/x86_64-unknown-linux-gnu/c++/v1 --libcxx-library-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib/x86_64-unknown-linux-gnu --triple x86_64-unknown-linux-gnu --build-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build --lldb-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build/module-cache-lldb/lldb-api --clang-module-cache-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/lldb-test-build/module-cache-clang/lldb-api --executable /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/lldb --lldb-python-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/local/lib/python3.12/dist-packages --compiler /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/clang --dsymutil /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./bin --lldb-obj-root /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb --lldb-libs-dir /home/gha/actions-runner/_work/llvm-project/llvm-project/build/./lib --cmake-build-type Release /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/API/functionalities/scripted_frame_provider/circular_dependency -p TestFrameProviderCircularDependency.py
--
Exit Code: 1

Command Output (stdout):
--
Skipping the following test categories: msvcstl, dsym, pdb, gmodules, debugserver, objc

FAILED (failures=1)

--
```
https://github.com/llvm/llvm-project/pull/210992#issuecomment-5037054976
https://github.com/llvm/llvm-project/pull/209781#issuecomment-4996585222
https://github.com/llvm/llvm-project/pull/208348#issuecomment-5108222325

Failed on the Arm linux buildbot too https://lab.llvm.org/buildbot/#/builders/18/builds/26361. Same error:
```
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-arm-ubuntu/build/bin/clang-arm) :: test_circular_dependency_handle_command_in_init (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase)
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-arm-ubuntu/build/bin/clang-arm) :: test_circular_dependency_with_function_replacement (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase)
======================================================================
FAIL: test_circular_dependency_evaluate_expression_in_get_frame (TestFrameProviderCircularDependency.FrameProviderCircularDependencyTestCase)
Test that calling EvaluateExpression in get_frame_at_index doesn't
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tcwg-buildbot/worker/lldb-arm-ubuntu/llvm-project/lldb/test/API/functionalities/scripted_frame_provider/circular_dependency/TestFrameProviderCircularDependency.py", line 204, in test_circular_dependency_evaluate_expression_in_get_frame
self.assertIsNotNone(
AssertionError: unexpectedly None : Frame 0 should have function name
Config=arm-/home/tcwg-buildbot/worker/lldb-arm-ubuntu/build/bin/clang
----------------------------------------------------------------------
```

Don't know how far the problem goes back.

I have had the problem locally on x86 but not been able to trigger it reliably.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

TestFrameProviderCircularDependency.py is flakey in GitHub CI

Description

Contributor guide

Assessment

Get new issues in your inbox