llvm / llvm/llvm-project

[BOLT] -frame-opt=hot removes required stack allocation, causing writes outside the x86-64 red zone

Open
#211,356 1 comment 0 reactions 0 assignees View on GitHub
BOLT miscompilation
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

When processing an x86-64 executable compiled with `-fno-omit-frame-pointer`, BOLT’s `-frame-opt=hot` can incorrectly remove a required stack allocation while retaining memory accesses relative to the frame pointer. This produces writes below the 128-byte stack red zone.

BOLT's allocation-combiner pass does not take into account %rbp-relative stack access when checking if coalescing can occur, only %rsp access is consider.

A small reproducer is:

``` C
#include
#include

__attribute__((noinline))
static uint64_t stack_example(const uint64_t input[20]) {
volatile uint64_t values[20];
uint64_t result = 0;

for (unsigned int i = 0; i < 20; i++) {
values[i] = input[i] + i;
result += values[i];
}

return result;
}

int main(void) {
uint64_t input[20] = {0};
uint64_t result = 0;

result = stack_example(input);
printf("%lu\n", result);
return 0;
}
```

```
# clang-22 -O3 -g -fno-omit-frame-pointer -Wl,--emit-relocs mre.c -o mre
# ./mre
190
# llvm-bolt-22 mre -frame-opt=hot -o bolt-hot
BOLT-INFO: shared object or position-independent executable detected
...
BOLT-INFO: Allocation combiner: 2 empty spaces coalesced (dyn count: 0).
BOLT-INFO: patched build-id (flipped last bit)
BOLT-INFO: setting _end to 0x400498
# valgrind --quiet --error-exitcode=13 --exit-on-first-error=yes ./bolt-hot
==145== Invalid write of size 8
==145== at 0x50818B: stack_example (in /root/bolt-hot)
==145== by 0x508147: main (in /root/bolt-hot)
==145== Address 0x1fff0002b0 is on thread 1's stack
==145== 160 bytes below stack pointer
==145==
==145==
==145== Exit program on first error (--exit-on-first-error=yes)
```

Running BOLT with `-frame-opt=none` does not cause this error:
```
# llvm-bolt-22 mre -frame-opt=none -o bolt-none
BOLT-INFO: shared object or position-independent executable detected
...
BOLT-INFO: setting _end to 0x4004a0
# valgrind --quiet --error-exitcode=13 --exit-on-first-error=yes ./bolt-none
190
```

In real code this invalid removal was found in the HACL hash functions in CPython with invalid memory access being detected by valgrind, astral-sh/python-build-standalone/issues/1186.

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the provided C program and the llvm-bolt commands, then inspect BOLT's allocation-combiner pass where stack accesses are checked. Ensure frame-opt=hot does not remove an allocation needed by %rbp-relative accesses, and verify the reproducer completes without Valgrind invalid-write errors while frame-opt=none remains unaffected.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.