llvm / llvm/llvm-project

[VectorCombine] Compose nested fixed shuffles through bitcasts

Open
#219,717 0 comments 0 reactions 1 assignee Claimed by @ParkHanbum View on GitHub
llvm:vectorcombine missed-optimization
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

LLVM does not compose the two fixed shuffle masks in this reproducer:

```llvm
define <8 x float> @poc(<32 x i8> %v) {
%bytes = shufflevector <32 x i8> %v, <32 x i8> poison, <32 x i32>
%half = bitcast <32 x i8> %bytes to <16 x i16>
%words = shufflevector <16 x i16> %half, <16 x i16> poison, <16 x i32>
%dwords = bitcast <16 x i16> %words to <8 x i32>
%signed = ashr <8 x i32> %dwords, splat (i32 24)
%result = sitofp <8 x i32> %signed to <8 x float>
ret <8 x float> %result
}
```

The expected form composes the masks directly:

```llvm
define <8 x float> @poc(<32 x i8> %v) {
%bytes = shufflevector <32 x i8> %v, <32 x i8> poison, <32 x i32>
%dwords = bitcast <32 x i8> %bytes to <8 x i32>
%signed = ashr <8 x i32> %dwords, splat (i32 24)
%result = sitofp <8 x i32> %signed to <8 x float>
ret <8 x float> %result
}
```

The source remains uncomposed after `opt -S -passes='default'`.

In the full case, four such chains feed identical `sitofp` and FMA consumers.
After this fold, existing O3 combines them into one `<128 x i8>` shuffle, one
wide `ashr`, one `<32 x i32>` conversion, and one `llvm.fma.v32f32`.

## Real-world corpus occurrence

This pattern was found in ONNX Runtime's AVX2 signed quantized-add kernel:

- Upstream source:
[`onnxruntime/core/mlas/lib/intrinsics/avx2/qladd_avx2.cpp`](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/mlas/lib/intrinsics/avx2/qladd_avx2.cpp)
- Function: `MlasQLinearAddS8KernelAvx2`

The source uses two levels of AVX2 unpacking, `MlasShiftRight24Epi32`,
and conversion to float. In the optimized IR generated for this function, that
produces 12 logical eight-lane instances of the nested-shuffle shape:

| Kernel path | Instances |
| --- | ---: |
| scalar B, widen Input A | 4 |
| vector B, widen Input B | 4 |
| vector B, widen Input A | 4 |

This is one production source file and one function, not a corpus-wide
frequency claim. The repeated instances are useful because applying the local
fold to all four groups exposes the larger existing O3 combine described above.

Backend results after final O3:

| Target | Static instructions | Block RThroughput | Result |
| --- | ---: | ---: | --- |
| x86-64-v4 | 17 -> 14 | 8.0 -> 4.0 | improves |
| x86-64 Haswell | 22 -> 20 | 8.0 -> 6.5 | improves; larger constants |
| RISC-V P670 RVV128 | 61 -> 26 | 44.0 -> 25.0 | improves |
| AArch64 Neoverse-V1 | 48 -> 52 | 8.0 -> 8.6 | regresses |

This fold therefore needs target-cost gating rather than unconditional
canonicalization.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.