[X86][WinEH] Win64 catchret continuation resumes with stale RSP after dynamic alloca + throwing invoke
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Description
Consider the following reduced IR:
```llvm
declare i32 @__CxxFrameHandler3(...)
declare void @opaque()
declare void @maythrow()
declare void @maythrow_big(i64, i64, i64, i64, i64, i64, i64, i64, i64)
declare ptr @llvm.stacksave.p0()
declare void @llvm.stackrestore.p0(ptr)
declare void @llvm.memset.p0.i64(ptr nocapture writeonly, i8, i64, i1)
define void @test(i64 %n) personality ptr @__CxxFrameHandler3 {
entry:
%sp = call ptr @llvm.stacksave.p0()
%buf = alloca i8, i64 %n, align 16
call void @llvm.memset.p0.i64(ptr %buf, i8 0, i64 %n, i1 false)
call void @llvm.stackrestore.p0(ptr %sp)
invoke void @maythrow() to label %exit unwind label %catch.dispatch
catch.dispatch:
%cs = catchswitch within none [label %catch] unwind to caller
catch:
%cp = catchpad within %cs [ptr null, i32 64, ptr null]
call void @opaque() [ "funclet"(token %cp) ]
catchret from %cp to label %cont
cont:
call void @opaque()
ret void
exit:
ret void
}
define void @test_manyargs(i64 %n) personality ptr @__CxxFrameHandler3 {
entry:
%sp = call ptr @llvm.stacksave.p0()
%buf = alloca i8, i64 %n, align 16
call void @llvm.memset.p0.i64(ptr %buf, i8 0, i64 %n, i1 false)
call void @llvm.stackrestore.p0(ptr %sp)
invoke void @maythrow_big(i64 0, i64 1, i64 2, i64 3, i64 4, i64 5, i64 6,
i64 7, i64 8)
to label %exit unwind label %catch.dispatch
catch.dispatch:
%cs = catchswitch within none [label %catch] unwind to caller
catch:
%cp = catchpad within %cs [ptr null, i32 64, ptr null]
call void @opaque() [ "funclet"(token %cp) ]
catchret from %cp to label %cont
cont:
call void @opaque()
ret void
exit:
ret void
}
```
On `x86_64-pc-windows-msvc`, the parent frame uses explicit `ADJCALLSTACKDOWN/UP` around the throwing `invoke` because the function has a dynamic alloca and therefore does not reserve a fixed outgoing call frame.
Before the matching `ADJCALLSTACKUP` runs, the exceptional edge transfers control into the funclet. `catchret` then resumes the continuation block without first restoring RSP to the parent frame's stable post-prologue value.
This means the continuation inherits the throwing invoke's transient call-frame adjustment.
In practice, a real C++ reproducer using `_alloca()` inside a loop accumulates stack drift on every throw/catch iteration:
```text
diff(0-100) = 3200
```
and if the throwing call needs an 80-byte outgoing call frame instead of 32 bytes, the drift scales accordingly:
```text
diff(0-100) = 8000
```
## Expected behavior
`catchret` should resume through a block that restores RSP to the parent frame's stable post-prologue value before executing the continuation block, independent of the outgoing call-frame size used by the throwing invoke.
## Actual behavior
The generated Win64 code resumes the continuation with the invoke's temporary call-frame adjustment still in effect, so the continuation runs with a stale RSP and repeated throws drift the stack.
## Assembly shape
For the 32-byte case, the generated parent frame looks like:
```asm
sub rsp, 32
call maythrow
add rsp, 32
```
but on the exceptional edge the `add rsp, 32` is not reached. The catch funclet returns directly to the continuation, so the continuation executes without normalizing RSP first.
The 80-byte case behaves the same way with `sub/add rsp, 80`.
## Root cause
This looks like a gap in how Win64 `catchret` continuations are modeled:
* the parent frame does not reserve a fixed call frame when dynamic allocas are present,
* the throwing `invoke` therefore uses explicit call-frame adjustments,
* the exceptional edge leaves before the matching destroy runs,
* but Win64 `catchret` continuations do not restore the parent stack state before resuming normal control flow.
The bug is not in funclet fixed-frame sizing. The leak size tracks the throwing invoke's outgoing call-frame size instead.
## Versions
I reproduced this with:
* LLVM/clang 19
* LLVM/clang 21
* LLVM/clang thunk
## Proposed fix
I have a local fix that:
* always routes Win64 `catchret` through a dedicated restore block, and
* restores RSP from the parent frame pointer's stable Win64 SEH frame offset before entering the continuation.
I also added a regression test covering both 32-byte and 80-byte outgoing call-frame sizes.
Contributor guide
Research direction
Start by reproducing the reduced LLVM IR on x86_64-pc-windows-msvc and inspect the generated assembly around the throwing invoke, catchret, and explicit ADJCALLSTACK adjustments. Compare the 32-byte and 80-byte cases. Done means catchret reaches the continuation with the parent frame's stable RSP and the regression test covers both outgoing call-frame sizes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100