llvm / llvm/llvm-project

[LNT] [VE] [Ptrdist/ft] Test-suite output hash mismatch when the graph RNG is run without a seed argument

Open
#222,582 0 comments 0 reactions 0 assignees View on GitHub
new issue
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Consider the test MultiSource/Benchmarks/Ptrdist/ft in llvm-test-suite
built and run with upstream clang (21.1.6, 22.1.8, 23.1.0) via lnt

the test fails the output comparison
=========================================================================
-- Testing: 1 tests, 1 workers --
FAIL: test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test (1 of 1)
******************** TEST 'test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test' FAILED ********************
/sandbox/tools/fpcmp-target: Comparison failed, textual difference between '4' and '6'

Input 1:
405c17bc7251b2cf20b56e716cb8bc35

Input 2:
63acb35e9d086e032ff07f8fdbe9d00a

********************
Failed Tests (1):
test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test
============================================================================

reproduces identically on clang 21.1.6, 22.1.8 and 23.1.0, and at -O0, -O2, -O3.

The output hash 405c17bc7251b2cf20b56e716cb8bc35 is the same across all three clang versions and all three optimization levels tested; only ft.reference_output (63acb35e9d086e032ff07f8fdbe9d00a) disagrees.

Contributor guide

Open the contributing guide

Research direction

Start with MultiSource/Benchmarks/Ptrdist/ft/ft.test in llvm-test-suite and reproduce it through lnt using the reported clang versions and optimization levels. Compare the generated output hash with ft.reference_output; done means the test-suite comparison passes without the mismatch reported by fpcmp-target.

Written by the indexing model from the issue text.

Assessment

Domain
compilers, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.