MachO_weak_references.s periodically failing
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
See, e.g., https://lab.llvm.org/buildbot/#/builders/52/builds/12009/steps/11/logs/stdio
Failing log is:
```
-- Testing: 92706 tests, 64 workers --
Testing: 0.. 10.. 20.. 30.. 40.. 50..
FAIL: LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s (56844 of 92706)
******************** TEST 'LLVM :: ExecutionEngine/JITLink/x86-64/MachO_weak_references.s' FAILED ********************
Exit Code: 1
Command Output (stdout):
--
# RUN: at line 1
rm -rf /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp && mkdir -p /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp
# executed command: rm -rf /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp
# note: command had no output on stdout or stderr
# executed command: mkdir -p /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp
# note: command had no output on stdout or stderr
# RUN: at line 2
/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s
# executed command: /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s
# note: command had no output on stdout or stderr
# RUN: at line 3
/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# executed command: /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# note: command had no output on stdout or stderr
# RUN: at line 4
/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# executed command: /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# note: command had no output on stdout or stderr
# error: command failed with exit status: 1
--
```
> LLVM Buildbot has detected a new failure on builder `sanitizer-x86_64-linux-bootstrap-asan` running on `sanitizer-buildbot1` while building `llvm` at step 2 "annotate".
>
> Full details are available at: https://lab.llvm.org/buildbot/#/builders/52/builds/12009
>
> Here is the relevant piece of the build log for the reference
This failure looks like a flaky test, rather than fallout from this commit.
Interestingly:
```
# RUN: at line 4
/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# executed command: /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/bin/llvm-jitlink -noexec -check-name=jitlink-check-bar-absent -check=/home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s /home/b/sanitizer-x86_64-linux-bootstrap-asan/build/llvm_build_asan/test/ExecutionEngine/JITLink/x86-64/Output/MachO_weak_references.s.tmp/macho_weak_refs.o
# note: command had no output on stdout or stderr
# error: command failed with exit status: 1
```
The test here is pretty trivial, and we're failing in the "bar absent" case:
```
# RUN: rm -rf %t && mkdir -p %t
# RUN: llvm-mc -triple=x86_64-apple-macosx10.9 -filetype=obj -o %t/macho_weak_refs.o %s
# RUN: llvm-jitlink -noexec -check-name=jitlink-check-bar-present -abs bar=0x1 -check=%s %t/macho_weak_refs.o
# RUN: llvm-jitlink -debug-only=jitlink,llvm_jitlink -num-threads=0 -noexec -check-name=jitlink-check-bar-absent -check=%s %t/macho_weak_refs.o
# Test weak reference handling by linking with and without a definition of 'bar' available.
.section __TEXT,__text,regular,pure_instructions
.build_version macos, 10, 14 sdk_version 10, 14
.globl _main
.p2align 4, 0x90
_main:
# jitlink-check-bar-present: *{8}(got_addr(macho_weak_refs.o, bar)) = bar
# jitlink-check-bar-absent: *{8}(got_addr(macho_weak_refs.o, bar)) = 0
cmpq $0, bar@GOTPCREL(%rip)
.weak_reference bar
.subsections_via_symbols
```
But in the case of an "ordinary" failure (i.e. bar being non-zero, or the JIT erroneously reporting it as such, or "bar" being periodically added via process symbols) then we'd expect output along the lines of:
```
# .---command stderr------------
# | Expression '*{8}(got_addr(macho_weak_refs.o, bar)) = 0' is false: 0xcafef00d != 0x0
# | /home/lhames/Projects/llvm-github/build/x86-64/Debug+Asserts/llvm-tmp/bin/llvm-jitlink: Some checks in /home/lhames/Projects/llvm-github/llvm-project/llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s failed
# `-----------------------------
# error: command failed with exit status: 1
```
but we're not seeing that. So whatever is doing this is causing us to exit without expected diagnostics, and without an obvious crash / backtrace.
The only obvious returns from `llvm-jitlink` with return code 1 are all proceeded with paths that should output some error.
If anyone sees this locally and spots anything that could explain this please share!
Contributor guide
Research direction
Start with llvm/test/ExecutionEngine/JITLink/x86-64/MachO_weak_references.s and reproduce the failure using the shown llvm-mc and llvm-jitlink commands, especially the bar-absent check. Compare the failing exit path with the expected diagnostics and investigate whether process symbols or another runtime condition is involved. Done means the cause is identified and the test no longer fails intermittently.
Written by the indexing model from the issue text.
Assessment
- Domain
- compilers, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100