llvm / llvm/llvm-project

[clang] flaky unit test TimeProfilerTest.ConstantEvaluationC99

Open
#196,220 0 comments 0 reactions 0 assignees View on GitHub
clang platform:macos third-party:unittests
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Occasionally (1 in 10 chance or so) this test fails for no reason, on a MacOS host:

```
******************** TEST 'Clang-Unit :: ./AllClangUnitTests/29/51' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:build/rwd/tools/clang/unittests/./AllClangUnitTests-Clang-Unit-52457-29-51.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=51 GTEST_SHARD_INDEX=29 build/rwd/tools/clang/unittests/./AllClangUnitTests
--

Script:
--
build/rwd/tools/clang/unittests/./AllClangUnitTests --gtest_filter=TimeProfilerTest.ConstantEvaluationC99
--
clang/unittests/Support/TimeProfilerTest.cpp:366: Failure
Expected equality of these values:
R"(
ExecuteCompiler
| Frontend (test.c)
| | ParseDeclarationOrFunctionDefinition (test.c:2:1)
| | | isIntegerConstantExpr ()
| | | EvaluateKnownConstIntCheckOverflow ()
| PerformPendingInstantiations
)"
Which is: "\nExecuteCompiler\n| Frontend (test.c)\n| | ParseDeclarationOrFunctionDefinition (test.c:2:1)\n| | | isIntegerConstantExpr ()\n| | | EvaluateKnownConstIntCheckOverflow ()\n| PerformPendingInstantiations\n"
buildTraceGraph(Json)
Which is: "\nExecuteCompiler\n| Frontend (test.c)\n| | ParseDeclarationOrFunctionDefinition (test.c:2:1)\n| | | isIntegerConstantExpr ()\n| | | EvaluateKnownConstIntCheckOverflow ()\n| | | PerformPendingInstantiations\n"
With diff:
@@ -5,3 +5,3 @@
| | | isIntegerConstantExpr ()
| | | EvaluateKnownConstIntCheckOverflow ()
-| PerformPendingInstantiations\n
+| | | PerformPendingInstantiations\n

clang/unittests/Support/TimeProfilerTest.cpp:366
Expected equality of these values:
R"(
ExecuteCompiler
| Frontend (test.c)
| | ParseDeclarationOrFunctionDefinition (test.c:2:1)
| | | isIntegerConstantExpr ()
| | | EvaluateKnownConstIntCheckOverflow ()
| PerformPendingInstantiations
)"
Which is: "\nExecuteCompiler\n| Frontend (test.c)\n| | ParseDeclarationOrFunctionDefinition (test.c:2:1)\n| | | isIntegerConstantExpr ()\n| | | EvaluateKnownConstIntCheckOverflow ()\n| PerformPendingInstantiations\n"
buildTraceGraph(Json)
Which is: "\nExecuteCompiler\n| Frontend (test.c)\n| | ParseDeclarationOrFunctionDefinition (test.c:2:1)\n| | | isIntegerConstantExpr ()\n| | | EvaluateKnownConstIntCheckOverflow ()\n| | | PerformPendingInstantiations\n"
With diff:
@@ -5,3 +5,3 @@
| | | isIntegerConstantExpr ()
| | | EvaluateKnownConstIntCheckOverflow ()
-| PerformPendingInstantiations\n
+| | | PerformPendingInstantiations\n
```

Contributor guide

Open the contributing guide

Research direction

Start with clang/unittests/Support/TimeProfilerTest.cpp at line 366 and run the TimeProfilerTest.ConstantEvaluationC99 filter repeatedly on a macOS host. Compare the expected trace with buildTraceGraph(Json), then investigate why PerformPendingInstantiations sometimes has a different nesting level. Done means the test no longer flakes and its trace remains consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.