llvm / llvm/llvm-project

[x86][CodeGen] Unnecessary spill of RSP to stack causes invalid frame tracking and stack loop in KMSAN build

Open
#206,973 2 comments 0 reactions 0 assignees View on GitHub
backend:X86 llvm:codegen
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

In Linux kernel builds with `CONFIG_KMSAN=y` and DWARF debug info enabled, Clang emits invalid/suboptimal stack pointer manipulations in functions under high register pressure (such as [`folio_zero_user()`](https://elixir.bootlin.com/linux/v7.1.2/source/mm/memory.c#L7445)).
See the discussion at https://lore.kernel.org/all/20260630104434.GC751831@noisy.programming.kicks-ass.net/T/#u

**Issue Description**
Under register pressure from KMSAN instrumentation, Clang spills %rsp to a stack slot and restores it later via a temporary register.

Disassembly pattern (courtesy of Peter Zijlstra):

```asm
mov %rsp, %rcx
1: mov %rcx, %rsp
...
mov %rsp, 0x68(%rsp) ; Spilling RSP to a stack slot
...
mov 0x68(%rsp), %rcx ; Reloading spilled RSP from stack slot
test ...
je 1b
mov %rcx, %r12
...
mov %r12, %rcx
jmp 1b
```

Linux `objtool` performs static unwinding analysis and cannot track register values spilled to memory slots. When `%rsp` is restored from `%rcx` (loaded from memory), `objtool` loses track of the CFA base register, leading to: error: `objtool: folio_zero_user+0x801: unknown CFA base reg -1`

Bypassing `objtool` verification [results in a kernel boot hang](https://lore.kernel.org/all/87tsqjq3i3.ffs@fw13/) inside the page fault handler due to broken stack frame state.

**To reproduce:**
[repro.c](https://github.com/user-attachments/files/29551821/repro.c)

Compile with:

```
clang -D__KERNEL__ --target=x86_64-linux-gnu -fintegrated-as \
-std=gnu11 -fshort-wchar -funsigned-char -fno-common -fno-PIE -fno-strict-aliasing \
-mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mno-sse4a -fcf-protection=branch \
-fno-jump-tables -m64 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mstack-alignment=8 \
-mskip-rax-setup -march=x86-64 -mtune=generic -mno-red-zone -mcmodel=kernel \
-mstack-protector-guard-reg=gs -mstack-protector-guard-symbol=__ref_stack_chk_guard \
-fno-asynchronous-unwind-tables -mretpoline-external-thunk -mindirect-branch-cs-prefix \
-mfunction-return=thunk-extern -fpatchable-function-entry=16,16 -fno-delete-null-pointer-checks \
-O2 -fstack-protector-strong -fomit-frame-pointer -fexperimental-late-parse-attributes \
-fno-stack-clash-protection -falign-functions=16 -fstrict-flex-arrays=3 -fms-extensions \
-fno-strict-overflow -fno-stack-check -fno-builtin-wcslen -g \
-fsanitize=kernel-memory -fsanitize-memory-param-retval \
-w -c -o repro.o repro.c
```

If you have a Linux kernel checkout, run objtool:

```
$ /path/to/linux/tools/objtool/objtool --hacks=jump_label --hacks=noinstr --hacks=skylake \
--ibt --orc --retpoline --rethunk --static-call --uaccess --prefix=16 \
--noinstr --unret --link repro.o
repro.o: warning: objtool: folio_zero_user+0x6f1: undefined stack state
repro.o: error: objtool: folio_zero_user+0x6f1: unknown CFA base reg -1
```

Otherwise, look at the objdump output:

```
$ llvm-objdump -d --no-show-raw-insn repro.o | grep -A 3 'movq.*\(%rsp,\|%rcx\|0x70(%rsp)\)'
...
--
6f0: movq %rsp, %rcx
6f3: testq %r12, %r12
6f6: jne 0x7c2
6fc: xorl %eax, %eax
6fe: movq %rcx, %rsp
701: movq %rbx, %rcx
704: movq %r14, %rdi
707: callq 0x70c
70c: shrq $0x3, %rcx
--
79f: movq %rsp, 0x70(%rsp)
7a4: movl 0x8(%rsp), %edi
7a8: callq 0x7ad
7ad: movl %eax, %edi
--
7b4: movq 0x70(%rsp), %rcx
7b9: testq %r12, %r12
7bc: je 0x6fc
7c2: movl 0x14(%rsp), %edi
```

Contributor guide

Open the contributing guide

Research direction

Start with repro.c and the provided Clang command, then inspect llvm-objdump output around folio_zero_user and run Linux tools/objtool/objtool with the shown options. Compare the generated stack-pointer sequence with objtool’s CFA diagnostics; done means the reproducer no longer triggers the reported invalid stack tracking and objtool failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
compilers, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.