Missed Optimization [WholeProgramDevirt] -O2 pass merges virtual dispatch sites, defeating WPD SingleImpl resolution
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Description
`-fwhole-program-vtables` with ThinLTO fails to devirtualize virtual calls at `-O2`/`-O3` when multiple virtual dispatch sites on the same base-class pointer are merged into a single indirect call with a computed vtable offset.
The pattern: a function switches on a runtime enum and dispatches a different virtual method per case — all through the same `Base*`. Each vtable slot has exactly one implementation program-wide (`SingleImpl`). WPD should resolve every slot to a direct call.
**WPD works at `-O1` but not at `-O2`.** At `-O1`, all 5 dispatch sites have fixed-offset GEPs and WPD resolves each via `SingleImpl`. At `-O2`, a pre-LTO optimization pass merges the 5 sites into a single indirect call with a variable-offset GEP (`vtable[base + enum*8]`). WPD cannot resolve a variable offset to a specific slot, so `SingleImpl` never fires.
This pattern is common in event dispatchers, protocol handlers, and state machines — anywhere a tagged enum selects which virtual callback to invoke.
**Godbolt (linked binary, `-O1` vs `-O2` — same flags, same code, different result):** https://godbolt.org/z/ojh4sjWxc
## Evidence: `-O1` vs `-O2` WPD remarks
$ clang++ -O1 -flto=thin -fwhole-program-vtables -fvisibility=hidden
-fuse-ld=lld -Rpass=wholeprogramdevirt -o test test.cpp
test.cpp:44:31: single-impl: devirtualized a call to TcpHandler::onError(Message const&)
test.cpp:43:31: single-impl: devirtualized a call to TcpHandler::onClose(Message const&)
test.cpp:42:31: single-impl: devirtualized a call to TcpHandler::onHeartbeat(Message const&)
test.cpp:41:31: single-impl: devirtualized a call to TcpHandler::onData(Message const&)
test.cpp:40:31: single-impl: devirtualized a call to TcpHandler::onConnect(Message const&)
$ clang++ -O2 -flto=thin -fwhole-program-vtables -fvisibility=hidden
-fuse-ld=lld -Rpass=wholeprogramdevirt -Rpass-missed=wholeprogramdevirt -o test test.cpp
(no output — WPD does not fire, no missed remarks either)
## Actual result (`-O3`)
`dispatch` in the linked binary:
```asm
dispatch(Handler*, Message const&):
test %rdi,%rdi
je return
mov 0x4(%rsi),%eax ; log() inlined (correct)
mov %eax,0x8(%rdi)
movzbl (%rsi),%eax ; load msg.type
cmp $0x4,%rax
ja return
mov (%rdi),%rcx ; load vtable pointer
mov 0x10(%rcx,%rax,8),%rax ; computed slot: vtable[2 + type]
jmp *%rax ; INDIRECT — not devirtualized
```
### Expected result
WPD should resolve each vtable slot to a direct call. Whether the compiler then inlines those direct calls is a separate decision. The minimum expected output is 5 direct
calls instead of 1 indirect call through a variable vtable offset:
```
switch (msg.type) →
case CONNECT: call TcpHandler::onConnect ; DIRECT
case DATA: call TcpHandler::onData ; DIRECT
case HEARTBEAT: call TcpHandler::onHeartbeat ; DIRECT
case CLOSE: call TcpHandler::onClose ; DIRECT
case ERROR: call TcpHandler::onError ; DIRECT
```
At -O1 this is exactly what happens — WPD resolves all 5 slots, then the inliner independently chooses to fold the small bodies into dispatch.
### Root cause
At -O1, the pre-LTO IR preserves 5 separate dispatch sites, each with a fixed-offset GEP and its own @llvm.type.test:
```llvm
; O1 IR — Case CONNECT (fixed offset 16):
%vt = load ptr, ptr %handler
%10 = call i1 @llvm.type.test(ptr %vt, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %10)
%slot = getelementptr inbounds i8, ptr %vt, i64 16 ; FIXED offset
%fptr = load ptr, ptr %slot
call void %fptr(ptr %handler, ptr %msg)
; O1 IR — Case DATA (fixed offset 24):
%vt2 = load ptr, ptr %handler
%11 = call i1 @llvm.type.test(ptr %vt2, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %11)
%slot2 = getelementptr inbounds i8, ptr %vt2, i64 24 ; FIXED offset
%fptr2 = load ptr, ptr %slot2
call void %fptr2(ptr %handler, ptr %msg)
; ... (3 more cases with offsets 32, 40, 48)
```
WPD resolves each {_ZTS7Handler, 16}, {_ZTS7Handler, 24}, etc. to SingleImpl → direct call.
At -O2, a per-TU optimization pass merges all 5 arms into one:
```asm
; O2 IR — single merged dispatch:
%offset = shl nuw nsw i8 %msg.type, 3 ; type * 8
%vt = load ptr, ptr %handler
%12 = call i1 @llvm.type.test(ptr %vt, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %12)
%ext = zext i8 %offset to i64
%base = getelementptr inbounds i8, ptr %vt, i64 %ext ; VARIABLE offset
%slot = getelementptr inbounds i8, ptr %base, i64 16
%fptr = load ptr, ptr %slot
call void %fptr(ptr %handler, ptr %msg) ; ONE indirect call
```
The @llvm.type.test survives, but the GEP offset is now variable. WPD maps {TypeID, ConstantByteOffset} to implementations — with a variable offset, it cannot determine which vtable slot is being called, so SingleImpl resolution does not fire.
### Impact
This pattern occurs wherever an enum/tag selects which virtual callback to invoke — event dispatchers, protocol handlers, state machines. These are typically hot paths in server and networking code. Devirtualization to direct calls eliminates dependent vtable loads and enables further interprocedural optimizations (inlining, constant propagation) that are impossible through indirect dispatch.
Contributor guide
Research direction
Start with the linked Godbolt reproduction and rerun the shown clang++ commands at -O1 and -O2 with ThinLTO, whole-program vtables, and WPD remarks enabled. Trace the WholeProgramDevirt handling of the merged variable-offset GEP described in the issue. Done means the enum-selected virtual calls can be resolved to five direct calls at -O2 or -O3 without regressing the existing -O1 behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100