MachineOutliner corrupts WinEH (SEH) tables for MSVC C++ EH funclets, producing invalid `.seh_startepilogue`/`.seh_endepilogue` sequences
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Title
MachineOutliner corrupts WinEH (SEH) tables for MSVC C++ EH funclets, producing invalid `.seh_startepilogue`/`.seh_endepilogue` sequences
## Environment
- clang version 22.1.3 (https://github.com/llvm/llvm-project e9846648fd6183ee6d8cbdb4502213fcf902a211)
- Target: `x86_64-pc-windows-msvc`
- Reproduces via both `clang-cl` and plain `clang++ --target=x86_64-pc-windows-msvc`
## Summary
With the machine outliner enabled (`-mllvm --enable-machine-outliner`) and no LTO, compiling
a small C++ file with two structurally-identical functions that each contain multiple local
objects with non-trivial (potentially-throwing) constructors fails with the integrated assembler
rejecting the emitted WinEH directives:
```
error: starting epilogue (.seh_startepilogue) before prologue has ended (.seh_endprologue) in ?make0@@YA?AUPair@@PEBD0@Z
error: Stray .seh_endepilogue in ?make0@@YA?AUPair@@PEBD0@Z
error: starting epilogue (.seh_startepilogue) before prologue has ended (.seh_endprologue) in ?dtor$30@?0??make0@@YA?AUPair@@PEBD0@Z@4HA
error: Stray .seh_endepilogue in ?dtor$30@?0??make0@@YA?AUPair@@PEBD0@Z@4HA
...
error: .seh_ directive must appear within an active frame
16 errors generated.
```
The errors hit both the two visible functions *and* their compiler-generated partial-construction
cleanup funclets (`dtor$N` — the per-scope unwind handlers MSVC C++ EH emits when an earlier local's
constructor must be undone if a later local's constructor throws). This strongly suggests the
outliner is extracting a code sequence that spans (or otherwise interacts badly with) a WinEH
funclet boundary, without correctly updating/duplicating the per-function SEH state-transition
metadata for the extracted/rewritten code.
- `--hot-cold-split` alone (without the outliner) does **not** reproduce this — isolated to
`--enable-machine-outliner` specifically.
- Reproduces at `-O1`, `-O2`, and `-Os`; does **not** reproduce at `-O0` (outlining presumably
doesn't kick in without any optimization).
- Requires **no** LTO and **no** project-specific flags beyond `/EHsc` (or `-fexceptions
-fcxx-exceptions`) + the outliner flag.
- We found this via a real-world build (an internal C++ codebase) where a build-config change
removed LTO from a couple of targets that happened to still have `--enable-machine-outliner`
enabled; under our normal (non-LTO-backend-forwarded) LTO configs the outliner pass never ran
against final WinEH-table-bearing code (our `-mllvm` enable is only passed at the front-end
compile step, never forwarded to the LTO backend link step), which is presumably why this had
gone unnoticed — losing LTO on those targets promptly hit this.
- **This is not LTO-mode-specific.** Forcing the same flag into the LTO backend itself (rather
than just the front-end compile step) reproduces the identical corruption under both ThinLTO
and full LTO:
```
clang++ --target=x86_64-pc-windows-msvc -fms-extensions -fexceptions -fcxx-exceptions \
-std=c++20 -O2 -flto=thin -fuse-ld=lld -Wl,-mllvm:--enable-machine-outliner \
minimal_main.cpp -o out.exe
```
This hits the exact same `.seh_startepilogue`/`Stray .seh_endepilogue` errors, this time inside
`libcpmt.lib`'s own EH vtable thunks (`std::exception`'s and `std::bad_alloc`'s scalar/vector
deleting dtors) once they get pulled into the same LTO merge as our TU. Same result with
`-flto=full`. So the bug is in the pass/codegen interaction itself, not something no-LTO-specific
— it just happens that most projects' LTO configs (like ours, until now) never forward
`-mllvm` outliner/hot-cold-split enables to the LTO backend, so it goes unnoticed there.
## Minimal reproducer (17 lines, self-contained, only depends on ``)
```cpp
#include
struct Str {
Str( const char * s ) : s_( s ) {}
Str( const Str & o ) : s_( o.s_ ) {}
std::string s_;
};
struct Pair {
Pair( Str const & a, Str const & b ) : a_( a ), b_( b ) {}
Str a_;
Str b_;
};
Pair make0( char const * x, char const * y ) { Str a( x ); Str b( y ); return Pair( a, b ); }
Pair make1( char const * x, char const * y ) { Str a( x ); Str b( y ); return Pair( a, b ); }
```
Command line:
```
clang++ -c --target=x86_64-pc-windows-msvc -fms-extensions -fexceptions -fcxx-exceptions \
-std=c++20 -O2 -mllvm --enable-machine-outliner minimal.cpp -o minimal.o
```
(equivalently: `clang-cl /c /EHsc /std:c++20 -O2 -mllvm --enable-machine-outliner minimal.cpp`)
A single copy of `make0` alone (no `make1`) does **not** reproduce — the outliner needs at least
two occurrences of the sequence to consider extracting it, consistent with this being a genuine
outlining-vs-WinEH interaction rather than something specific to one function's shape.
## Expected behavior
The file should compile cleanly (as it does at `-O0`, or with the outliner disabled at any
optimization level).
## Notes
- 100% deterministic across repeated runs on the same machine.
- Not investigated: whether this is specific to the MSVC C++ EH model (`/EHsc`, WinEH funclets)
vs. also reproducible with Itanium EH tables on a non-Windows triple — have not tried a
Linux/macOS target with the outliner enabled + LTO disabled + this same code shape.
Contributor guide
Research direction
Start by running the provided 17-line reproducer with x86_64-pc-windows-msvc, optimization, and --enable-machine-outliner. Inspect the MachineOutliner interaction with WinEH funclets and their SEH state-transition metadata. Done means the reproducer compiles cleanly with the outliner enabled and a regression test covers the failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100