llvm / llvm/llvm-project

[InstCombine] Sparse shuffle+bitcast+shift widening DAG is not folded to sext/zext

Open
#220,504 0 comments 0 reactions 0 assignees View on GitHub
llvm:instcombine missed-optimization
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

The optimized ONNX Runtime MLAS QLADD IR contains a byte-to-`i32` widening DAG expressed as repeated byte/word shuffles, bitcasts, and a constant shift. `default` leaves the DAG in place. For the signed form, the DAG is equivalent to `sext <8 x i8> -> <8 x i32>`; the unsigned form is the corresponding `lshr`/`zext` case. The normalized slice is measurable on AArch64, but its modeled block throughput is worse there (`1.0 -> 1.5` on `neoverse-v1`), so the ARM64 result is mixed rather than a speedup claim.

The corpus evidence below is a focused scan of one optimized IR file, not a claim about the full benchmark corpus.

## Reproducer

The following is a normalized full-lane POC extracted from `onnxruntime/core/mlas/lib/qladd.cpp`, `MlasQLinearAddS8Kernel` at line 109. In the corpus, the source bytes first come from a `<2 x i64>` load bitcast to `<16 x i8>`; only the low eight bytes feed this widening DAG, so the POC exposes those bytes directly as `<8 x i8>`.
```llvm
; S8 full 8-lane, noundef input from load i64 -> <8 x i8>
define <8 x i32> @src(<8 x i8> noundef %a) {
%shuffle1 = shufflevector <8 x i8> %a, <8 x i8> poison, <16 x i32>

%bc1 = bitcast <16 x i8> %shuffle1 to <8 x i16>
%sh = shufflevector <8 x i16> %bc1, <8 x i16> poison, <16 x i32>

%bc2 = bitcast <16 x i16> %sh to <8 x i32>
%r = ashr <8 x i32> %bc2, splat (i32 24)
ret <8 x i32> %r
}

; Expected (S8):
define <8 x i32> @tgt(<8 x i8> noundef %a) {
%r = sext <8 x i8> %a to <8 x i32>
ret <8 x i32> %r
}
```

For the U8 kernels, the final `ashr` is `lshr` and the expected operation is `zext`.

## Corpus Search Record

The focused exact-mask scan covered one optimized IR file, not the full benchmark corpus:

| File | Function | Matches |
| --- | --- | ---: |
| `onnxruntime__core__mlas__lib__qladd.cpp.ll` | `MlasQLinearAddS8Kernel` | 6 |
| `onnxruntime__core__mlas__lib__qladd.cpp.ll` | `MlasQLinearAddU8Kernel` | 6 |
| **Total** | | **12** |

The source/current-O3/replay-O3 exact-mask counts were `12/12/12`; parser errors were 0.

## Cost Evidence

`llc` plus `llvm-mca` was run on the `default` POC forms for 100 iterations (`x86_64-unknown-linux-gnu -mcpu=x86-64-v4`, `riscv64-unknown-linux-gnu -mcpu=sifive-p670`, and `aarch64-unknown-linux-gnu -mcpu=neoverse-v1`). These are modeled instruction/cycle costs, not an end-to-end QLADD application benchmark.
The instruction and uOp totals are the block totals reported by `llvm-mca` for the 100-iteration sequence, including the return instruction.

| Target | Source -> target instructions | Source -> target uOps | Source -> target cycles | Block RThroughput |
| --- | ---: | ---: | ---: | ---: |
| x86-64-v4 | 500 -> 200 (-60%) | 700 -> 400 (-42.9%) | 603 -> 303 (-49.8%) | 3.0 -> 1.0 (3.0x) |
| RISC-V P670 | 900 -> 400 (-55.6%) | 900 -> 400 (-55.6%) | 1804 -> 404 (-77.6%) | 3.0 -> 1.5 (2.0x) |
| AArch64 Neoverse-V1 (normalized slice) | 600 -> 400 (-33.3%) | 600 -> 400 (-33.3%) | 603 -> 403 (-33.2%) | 1.0 -> 1.5 (worse) |

The x86 assembly text artifacts were 677 -> 442 bytes; this is supplementary file size, not a measured machine-code text-section size.
The secondary low-lane U8 slice on AArch64 also reduces instructions/cycles (`400 -> 300`, `603 -> 403`) while worsening modeled `Block RThroughput` (`0.5 -> 1.0`).

The complete optimized QLADD module is not directly usable as an unchanged AArch64 benchmark because it contains x86-specific intrinsics such as `llvm.x86.sse2.cvtps2dq` and `llvm.x86.sse2.pack*`. The AArch64 row above therefore applies only to the target-independent normalized widening slice. It shows fewer instructions and lower total modeled cycles, but worse `Block RThroughput`; no broad ARM64 speedup claim is made.

Proof: https://alive2.llvm.org/ce/z/MoQm-3

Contributor guide

Open the contributing guide

Research direction

Start with the supplied LLVM IR @src/@tgt reproducer and run it through default to confirm the shuffle, bitcast, and shift DAG remains. Read the InstCombine implementation and compare the signed ashr case with the unsigned lshr case, using the qladd.cpp source context and Alive2 proof as references. Done means both forms fold to the expected sext/zext behavior without changing semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.