llvm / llvm/llvm-project

Strange Stack-Alignment Code Generated by Clobbering `esp` Can Overrite Function Return

Open
#188,670 2 comments 0 reactions 0 assignees View on GitHub
backend:X86 inline-asm
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

The following C code:
```c
#include
#include
#include

void __attribute__((optnone)) foo(void *a, uint32_t b) { return; }

uint32_t __attribute__((force_align_arg_pointer)) bar(void *a, uint32_t *b,
uint32_t c) {
asm("" : : : "eax", "esi", "ecx", "edx", "edi", "esp");

if (b || (a && c)) {
foo(a, c);
return 1;
}
return 0;
}

int __attribute__((optnone)) main() {
assert(bar(NULL, NULL, 0) == 0);
return 0;
}
```
when compiled for `x86_32` with `clang 22.1.1` trips the assertion on `bar` in `main`:
```sh
> clang -m32 -O1 test.c
> ./a.out
a.out: test.c:19: int main(): Assertion `bar(NULL, NULL, 0) == 0' failed.
Aborted (core dumped) ./a.out
```
`bar` as compiled always returns a garbage stack address rather than the expected return of `0`. This occurs for all inputs (not just the ones hard-coded in the harness). This is due to to a 4 byte spill and load of the saved value of `%esp` (technically 4 above `%esp` but it's used to restore `%esp`), for which `%eax` is chosen as the target register:
```asm
bar: # @bar
.cfi_startproc
# %bb.0:
leal 4(%esp), %eax # <--- %eax is used to store location of arguments
.cfi_def_cfa %eax, 0
andl $-16, %esp # <--- stack is aligned before stack frame is created
pushl -4(%eax)
pushl %ebp
movl %esp, %ebp
.cfi_escape 0x10, 0x05, 0x02, 0x75, 0x00 #
pushl %ebx
pushl %edi
pushl %esi
subl $28, %esp
movl %eax, -28(%ebp) # 4-byte Spill <-- store the argument location on the stack

# ... most of function body omitted for clarity

subl $8, %esp
pushl %edi
pushl %esi
calll foo
addl $16, %esp
movl $1, %eax
.LBB1_3: # <--- final basic block
movl -28(%ebp), %eax # 4-byte Reload <--- overwrite return code with saved %esp offset
addl $28, %esp
popl %esi
popl %edi
popl %ebx
popl %ebp
leal -4(%eax), %esp # <-- use saved value to restore %esp
.cfi_def_cfa %esp, 4
retl
```
Note that changing the function body or deleting clobbered registers from the `asm` block (excluding `%esp`) alters register selection and will produce a working function that still has this (in my opinion) weird code generation for managing `%esp`.

`clang` emits no warnings for this abuse of clobbers--this was originally minified from a bug where "what a clobber does" was misunderstood--but `gcc` warns that inline assembly cannot alter `%esp`. It also compiles the code and does not trip the assertion (with the `optnone` attribute swapped out with `optimize("-o0")` to be safe), though it may just be deciding the assembly is invalid.

I'd expect `clang` to issue a warning if any inline assembly tries to clobber `%esp` and I would not expect the stack frame to be created after alignment; in all other cases I've seen a frame be created first and then have the stack aligned afterwards, so it's still possible to find the old value of `%esp` off of `%ebp`.

On `clang` version `22.1.1`, `%eax` is selected as the register for this behavior on the peephole optimization pass. On an earlier compiler version, I saw this occurring on the previous machine-code sinking pass instead. I have tried and failed to set up `bugpoint` to reproduce this to provide a better idea of what is causing the issue, but I suspect it is down to final register allocation; it seems like the compiler loses track of `%eax` as a live value at the end of the function?

Notably, when re-testing this on the latest version of clang, I had to add additional registers to the inline assembly block to make this work (my original reproducer only had `asm("" : : : "eax", "esi", "ecx", "esp");`. I would guess that there are other register/function body combinations that would also work and it is just down to whether the compiler decides that `%eax` is the best choice for the purpose. The particular function body of `bar` is just the one that I got from reducing the original problematic code down until removing anything caused the bug to go away.

I've also uploaded the textual IR form (generated from `clang -m32 -O1 -emit-llvm -S` as [test.ll.txt](https://github.com/user-attachments/files/26259383/test.ll.txt) (to get GitHub to accept it) and from inspection I can't see anything in the IR that would cause something like this? I am no expert here but I expect that this happens after IR level optimization, which is why I'm leaning towards the backend potentially introducing the behavior.

Please let me know if I can provide any additional information, I haven't submitted a bug to `llvm` before and the [How to submit a bug](https://llvm.org/docs/HowToSubmitABug.html) page just says to fill out a form that no longer exists after the move to GitHub so I'm worried I'm missing some information.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue from test.c with clang -m32 -O1 and inspect the generated assembly alongside test.ll.txt. Trace the x86_32 backend handling of the %esp clobber, stack alignment, and the final return value. Done should include a minimized regression test and a confirmed resolution for the invalid return or an appropriate diagnostic.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.