llvm / llvm/llvm-project

Test failures under LLVM_REVERSE_ITERATION=On

Open
#214,872 1 comment 0 reactions 0 assignees View on GitHub
llvm test-suite
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Under reverse iteration in an asserts build, I see the following tests failing when running `check-llvm`. This is happening at least as of cb43aac8f5b8aac8e74ea47fa637cdeb81e025b2. I haven't yet tried other targets, like clang, lld, or runtimes. Setting LLVM_REVERSE_ITERATION=Off causes all tests to pass.

```
Failed Tests (4):
LLVM :: ExecutionEngine/RuntimeDyld/ARM/MachO_ARM_PIC_relocations.s
LLVM :: Transforms/LoopVectorize/AArch64/cmp_cost.ll
LLVM :: Transforms/SampleProfile/pseudo-probe-stale-profile-orphan-conflict.ll
LLVM :: tools/llvm-exegesis/X86/analysis-malformed-yaml-error.test
```
This seems odd, as I see the expensive checks buildbots passing: https://lab.llvm.org/buildbot/#/builders/31

These are pretty different parts of the codebase, so it seems unlikely that anything common between them was nondeterministic but also not causing issues elsewhere. I've only just found this and haven't investigated further through bisection or other means.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing `check-llvm` in an asserts build with `LLVM_REVERSE_ITERATION=On`, then inspect the four failing tests: the ARM RuntimeDyld relocation, AArch64 loop-vectorization cost, SampleProfile stale-profile case, and llvm-exegesis malformed-YAML test. Compare with `LLVM_REVERSE_ITERATION=Off` and determine whether the failures share a cause; done means identifying and fixing the failures or narrowing them to actionable independent issues.

Written by the indexing model from the issue text.

Assessment

Domain
compilers, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.