Missed interprocedural finite-set specialization: `{2,4}` helper call reduces to `select(0x88, 0xaa)`
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Summary
A caller has a runtime `num_bits` value, but a dominating switch proves that a
shared block is reached only for `num_bits == 2` or `num_bits == 4`. The block
computes a one-hot default zero point and calls a separate generic helper:
```text
zero_point = 1 << (num_bits - 1)
packed = GetPackedZeroPointValue(num_bits, zero_point)
```
The helper validates `num_bits`, has a throwing invalid path, and replicates the
field across one byte. For the two caller-proven values, its result is fixed:
| `num_bits` | `zero_point` | packed result |
| ---: | ---: | ---: |
| 2 | `0x02` | `0xaa` |
| 4 | `0x08` | `0x88` |
Current O3 retains the helper call at both eligible production call sites. The
desired caller specialization is:
```llvm
%is4 = icmp eq i64 %num_bits, 4
%packed = select i1 %is4, i8 -120, i8 -86 ; 0x88 or 0xaa
```
This is not a request to change the helper for arbitrary callers or inputs.
The helper's invalid behavior remains part of its general contract. The
opportunity is to prove that the invalid edge is unreachable from these
callers and evaluate the successful result for the finite input set.
## Real source origin
The pattern comes from ONNX Runtime's generic
[`GetPackedZeroPointValue`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cpu/moe/moe_quantization_cpu.cc#L24-L34)
helper. It enforces that `num_bits` is a positive divisor of 8, then repeatedly
ORs a masked field at `i * num_bits` offsets.
The caller's generic 2/4-bit path constructs
`default_zero_point = 1 << (num_bits - 1)` and calls the helper; see
[`DequantizeBlockWithMlas`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cpu/moe/moe_quantization_cpu.cc#L395-L399).
## Production IR shape
The important point is that the switch and pack are not initially in one
function. The caller looks like:
```llvm
switch i64 %num_bits, label %other [
i64 8, label %eight_bit
i64 4, label %two_or_four
i64 2, label %two_or_four
]
two_or_four:
%k32 = trunc nuw nsw i64 %num_bits to i32
%one_less = add nsw i32 %k32, -1
%one = shl nuw nsw i32 1, %one_less
%zero_point = trunc nuw nsw i32 %one to i8
%packed = tail call fastcc i8 @GetPackedZeroPointValue(
i64 %num_bits, i8 %zero_point)
; use %packed ...
```
The separate helper begins with the actual validation/exception boundary:
```llvm
define internal fastcc i8 @GetPackedZeroPointValue(
i64 range(i64 9, 8) %num_bits, i8 %zero_point)
personality ptr @__gxx_personality_v0 {
entry:
%minus_one = add i64 %num_bits, -1
%in_range = icmp ult i64 %minus_one, 8
br i1 %in_range, label %check_divisor, label %throw
check_divisor:
%k8 = trunc nuw nsw i64 %num_bits to i8
%rem = urem i8 8, %k8
%lanes = udiv i8 8, %k8
%divides = icmp eq i8 %rem, 0
br i1 %divides, label %pack, label %throw
throw:
; exception allocation, stack trace/message construction, and __cxa_throw
; ...
unreachable
pack:
; broadcast field and num_bits, shift lanes by 0*k ... 7*k,
; truncate to bytes, mask lanes >= 8/k
%result = tail call i8 @llvm.vector.reduce.or.v8i8(<8 x i8> %active)
ret i8 %result
}
```
On x86-64-v4, the helper uses vector variable shifts and a reduction. Generic
x86-64 and arm64 builds retain scalar-unrolled forms of the same source
algorithm.
The helper is `inlinehint`, but current inliner remarks at both production call
sites are:
```text
too costly to inline (cost=610, threshold=569)
```
The argument's wrapping range attribute is also too coarse to represent the
disjoint value set `{2,4}`. O3 consequently keeps the calls, both validity
checks, division, and generic pack.
## Proposed transformation boundary
Specialize only the shared caller block:
```llvm
two_or_four:
%is4 = icmp eq i64 %num_bits, 4
%packed = select i1 %is4, i8 -120, i8 -86
; use %packed ...
```
For this production module, the helper is internal and has exactly two calls;
both calls have the same proven 2/4 relation. After specializing both callers,
ordinary dead-code elimination can also remove the unused helper and its cold
exception machinery. That module-wide deletion is a consequence, not a
legality precondition: if another caller exists, the generic helper must remain.
## Literal constants are already handled
Direct literal calls are not the missed case. Current O3 folds them to:
```llvm
define i8 @constant_two() {
ret i8 -86 ; 0xaa
}
define i8 @constant_four() {
ret i8 -120 ; 0x88
}
```
This control was rerun with the same mechanically extracted production helper,
including its validation and C++ exception CFG; it is not an arithmetic-only
stand-in. O3 removes the helper and retains only the two constant returns.
The missing capability is to carry a caller-proven finite set across the call
boundary and materialize a select of the per-member constant evaluations.
## Current O3 and backend cost
A production-shaped standalone module preserves the actual helper validation,
C++ exception CFG, vector body, internal linkage, and two reduced caller
shells. No `noinline`, `llvm.assume`, or artificial side effect is used. Under
`default,verify`, source retains two helper calls and target retains the two
selects; the target's now-unused internal helper is removed.
Canonical x86-64-v4 LLVM-MCA results use 100 iterations:
| Target model | Variant | Instructions | Total cycles | Total uOps | Block RThroughput |
| --- | --- | ---: | ---: | ---: | ---: |
| x86-64-v4 | faithful source module | 11600 | 8399 | 18800 | 31.3 |
| x86-64-v4 | faithful proposed module | 2600 | 710 | 3600 | 6.0 |
Additional exact-module metrics:
| Metric | Source | Proposed |
| --- | ---: | ---: |
| ELF `.text` bytes | 473 | 85 |
| Disassembled instruction lines | 118 | 27 |
| Helper calls after O3 | 2 | 0 |
LLVM-MCA does not model calls or C++ exception control flow dynamically; it
assigns calls a synthetic latency and ignores program-counter updates. A
supplemental composed slice therefore keeps the caller switch, exact validity
checks, explicit noreturn invalid edge, and full vector pack while excluding
the interprocedural call and verbose exception construction:
| Target model | Variant | Instructions | Total cycles | Total uOps | Block RThroughput |
| --- | --- | ---: | ---: | ---: | ---: |
| x86-64-v4 | composed source slice | 3600 | 2190 | 5000 | 10.0 |
| x86-64-v4 | composed proposed slice | 1000 | 285 | 1200 | 2.0 |
The slice's `.text` is 149→32 bytes and its disassembled instruction count is
36→10. O3 still retains the divisibility check, cold invalid call, and vector
pack in the source slice. Thus the profitability result does not depend only
on dead-deleting the cold C++ exception machinery.
## Semantic validation and exception behavior
Cost was checked before semantic validation.
A direct Alive2 comparison of the separate-function IR is not a proof: Alive2
abstracts the internal callee as a declaration in the selected caller and
reports that the source call may not return. I am not counting that result as a
pass.
Instead, the caller-reachable helper semantics were composed explicitly in one
function: caller switch and fallback, both validity checks, a noreturn invalid
edge, the one-hot value, and the full vector pack. Strict bidirectional Alive2
then reports:
```text
1 correct transformations
0 incorrect transformations
0 failed-to-prove transformations
0 Alive2 errors
```
Two independent bidirectional lemmas report the same passing counters:
1. the 2/4 switch-valid domain always satisfies
`num_bits > 0 && num_bits <= 8 && 8 % num_bits == 0`;
2. the successful one-hot repeated pack equals the `0xaa`/`0x88` select.
No `llvm.assume` is used. In the actual helper, all externally observable
exception construction is dominated by validation failure. Before the checks,
the helper performs only stack allocation and a store to a nonescaping stack
slot; the successful path otherwise contains arithmetic and the reduction
before returning. Therefore the eligible caller replacement does not remove a
reachable throw or another visible successful-path effect.
This is a composed caller-context proof, not a claim that Alive2 directly
verified the untouched interprocedural pair or all C++ exception cleanup paths.
Contributor guide
Research direction
Start with the production-shaped standalone LLVM IR module and run default,verify, comparing retained helper calls with the proposed selects. Read the caller and helper origins in onnxruntime/contrib_ops/cpu/moe/moe_quantization_cpu.cc; no specific LLVM pass or test is named here. Done means eligible 2/4 calls become the 0x88/0xaa select, invalid behavior remains unreachable in context, and the helper is removed only when unused.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100