llvm / llvm/llvm-project

Missed Optimization [WholeProgramDevirt] -O2 pass merges virtual dispatch sites, defeating WPD SingleImpl resolution

Open
#185,811 7 comments 0 reactions 0 assignees View on GitHub
llvm:optimizations missed-optimization
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Description

`-fwhole-program-vtables` with ThinLTO fails to devirtualize virtual calls at `-O2`/`-O3` when multiple virtual dispatch sites on the same base-class pointer are merged into a single indirect call with a computed vtable offset.

The pattern: a function switches on a runtime enum and dispatches a different virtual method per case — all through the same `Base*`. Each vtable slot has exactly one implementation program-wide (`SingleImpl`). WPD should resolve every slot to a direct call.

**WPD works at `-O1` but not at `-O2`.** At `-O1`, all 5 dispatch sites have fixed-offset GEPs and WPD resolves each via `SingleImpl`. At `-O2`, a pre-LTO optimization pass merges the 5 sites into a single indirect call with a variable-offset GEP (`vtable[base + enum*8]`). WPD cannot resolve a variable offset to a specific slot, so `SingleImpl` never fires.

This pattern is common in event dispatchers, protocol handlers, and state machines — anywhere a tagged enum selects which virtual callback to invoke.

**Godbolt (linked binary, `-O1` vs `-O2` — same flags, same code, different result):** https://godbolt.org/z/ojh4sjWxc

## Evidence: `-O1` vs `-O2` WPD remarks

$ clang++ -O1 -flto=thin -fwhole-program-vtables -fvisibility=hidden
-fuse-ld=lld -Rpass=wholeprogramdevirt -o test test.cpp

test.cpp:44:31: single-impl: devirtualized a call to TcpHandler::onError(Message const&)
test.cpp:43:31: single-impl: devirtualized a call to TcpHandler::onClose(Message const&)
test.cpp:42:31: single-impl: devirtualized a call to TcpHandler::onHeartbeat(Message const&)
test.cpp:41:31: single-impl: devirtualized a call to TcpHandler::onData(Message const&)
test.cpp:40:31: single-impl: devirtualized a call to TcpHandler::onConnect(Message const&)

$ clang++ -O2 -flto=thin -fwhole-program-vtables -fvisibility=hidden
-fuse-ld=lld -Rpass=wholeprogramdevirt -Rpass-missed=wholeprogramdevirt -o test test.cpp

(no output — WPD does not fire, no missed remarks either)

## Actual result (`-O3`)

`dispatch` in the linked binary:

```asm
dispatch(Handler*, Message const&):
test %rdi,%rdi
je return
mov 0x4(%rsi),%eax ; log() inlined (correct)
mov %eax,0x8(%rdi)
movzbl (%rsi),%eax ; load msg.type
cmp $0x4,%rax
ja return
mov (%rdi),%rcx ; load vtable pointer
mov 0x10(%rcx,%rax,8),%rax ; computed slot: vtable[2 + type]
jmp *%rax ; INDIRECT — not devirtualized
```
### Expected result

WPD should resolve each vtable slot to a direct call. Whether the compiler then inlines those direct calls is a separate decision. The minimum expected output is 5 direct
calls instead of 1 indirect call through a variable vtable offset:
```
switch (msg.type) →
case CONNECT: call TcpHandler::onConnect ; DIRECT
case DATA: call TcpHandler::onData ; DIRECT
case HEARTBEAT: call TcpHandler::onHeartbeat ; DIRECT
case CLOSE: call TcpHandler::onClose ; DIRECT
case ERROR: call TcpHandler::onError ; DIRECT
```
At -O1 this is exactly what happens — WPD resolves all 5 slots, then the inliner independently chooses to fold the small bodies into dispatch.

### Root cause

At -O1, the pre-LTO IR preserves 5 separate dispatch sites, each with a fixed-offset GEP and its own @llvm.type.test:
```llvm
; O1 IR — Case CONNECT (fixed offset 16):
%vt = load ptr, ptr %handler
%10 = call i1 @llvm.type.test(ptr %vt, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %10)
%slot = getelementptr inbounds i8, ptr %vt, i64 16 ; FIXED offset
%fptr = load ptr, ptr %slot
call void %fptr(ptr %handler, ptr %msg)

; O1 IR — Case DATA (fixed offset 24):
%vt2 = load ptr, ptr %handler
%11 = call i1 @llvm.type.test(ptr %vt2, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %11)
%slot2 = getelementptr inbounds i8, ptr %vt2, i64 24 ; FIXED offset
%fptr2 = load ptr, ptr %slot2
call void %fptr2(ptr %handler, ptr %msg)

; ... (3 more cases with offsets 32, 40, 48)
```
WPD resolves each {_ZTS7Handler, 16}, {_ZTS7Handler, 24}, etc. to SingleImpl → direct call.

At -O2, a per-TU optimization pass merges all 5 arms into one:
```asm
; O2 IR — single merged dispatch:
%offset = shl nuw nsw i8 %msg.type, 3 ; type * 8
%vt = load ptr, ptr %handler
%12 = call i1 @llvm.type.test(ptr %vt, metadata !"_ZTS7Handler")
call void @llvm.assume(i1 %12)
%ext = zext i8 %offset to i64
%base = getelementptr inbounds i8, ptr %vt, i64 %ext ; VARIABLE offset
%slot = getelementptr inbounds i8, ptr %base, i64 16
%fptr = load ptr, ptr %slot
call void %fptr(ptr %handler, ptr %msg) ; ONE indirect call
```
The @llvm.type.test survives, but the GEP offset is now variable. WPD maps {TypeID, ConstantByteOffset} to implementations — with a variable offset, it cannot determine which vtable slot is being called, so SingleImpl resolution does not fire.

### Impact

This pattern occurs wherever an enum/tag selects which virtual callback to invoke — event dispatchers, protocol handlers, state machines. These are typically hot paths in server and networking code. Devirtualization to direct calls eliminates dependent vtable loads and enables further interprocedural optimizations (inlining, constant propagation) that are impossible through indirect dispatch.

Contributor guide

Open the contributing guide

Research direction

Start with the linked Godbolt reproduction and rerun the shown clang++ commands at -O1 and -O2 with ThinLTO, whole-program vtables, and WPD remarks enabled. Trace the WholeProgramDevirt handling of the merged variable-offset GEP described in the issue. Done means the enum-selected virtual calls can be resolved to five direct calls at -O2 or -O3 without regressing the existing -O1 behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.