SLP costs broadcast shuffles that are folded
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
RISC-V has `vfoo.vx` instructions , which splat a scalar in the second operand. The SLP cost model doesn't account for this though and we end up overcosting by an insertelement + broadcast shuffle:
```llvm
; RUN: opt -p slp-vectorizer -S -o - scratch/slp-splat-cost.ll -mtriple riscv64 -mattr=+v -debug 2<&1 | FileCheck %s
define void @fmul_splat(i32 %s, ptr %p, ptr %q) {
%x0 = load i32, ptr %p
%p1 = getelementptr inbounds nuw i32, ptr %p, i64 1
%x1 = load i32, ptr %p1
%p2 = getelementptr inbounds nuw i32, ptr %p, i64 2
%x2 = load i32, ptr %p2
%p3 = getelementptr inbounds nuw i32, ptr %p, i64 3
%x3 = load i32, ptr %p3
%m0 = add i32 %s, %x0
%m1 = add i32 %s, %x1
%m2 = add i32 %s, %x2
%m3 = add i32 %s, %x3
store i32 %m0, ptr %q
%q1 = getelementptr i32, ptr %q, i64 1
store i32 %m1, ptr %q1
%q2 = getelementptr i32, ptr %q, i64 2
store i32 %m2, ptr %q2
%q3 = getelementptr i32, ptr %q, i64 3
store i32 %m3, ptr %q3
ret void
}
```
```
SLP: Calculated costs for Tree:
1.
Operand 0:
i32 %s
i32 %s
i32 %s
i32 %s
Operand 1:
%x0 = load i32, ptr %p, align 4
%x1 = load i32, ptr %p1, align 4
%x2 = load i32, ptr %p2, align 4
%x3 = load i32, ptr %p3, align 4
Scalars:
%m0 = add i32 %s, %x0
%m1 = add i32 %s, %x1
%m2 = add i32 %s, %x2
%m3 = add i32 %s, %x3
State: Vectorize
MainOp: %m0 = add i32 %s, %x0
AltOp: %m0 = add i32 %s, %x0
VectorizedValue: NULL
ReuseShuffleIndices: Empty
ReorderIndices:
UserTreeIndex: {User:0 EdgeIdx:0}
SLP: Costs:
SLP: ReuseShuffleCost = 0
SLP: VectorCost = 1
SLP: ScalarCost = 4
SLP: ReuseShuffleCost + VecCost - ScalarCost = -3
SLP: Adding cost -3 for bundle Idx: 1, n=4 [ %m0 = add i32 %s, %x0, ..].
SLP: Current total cost = -6
SLP: Adding cost 2 for bundle Idx: 2, n=4 [i32 %s, ..]. <------ # THIS SHOULD BE FREE
SLP: Current total cost = -4
```
I believe this overcosting is contributing to the performance regressions we're seeing with SLP whenever it's moved after unrolling, which causes it to stop vectorizing, see https://github.com/llvm/llvm-project/pull/135318#issuecomment-5099702573
I'm not sure what the correct TTI API should look like. We have `canSplatOperand(unsigned Opcode, int Operand)` in RISCVTTIImpl but it's not a generic TTI hook.
cc @bababuck @alexey-bataev
Contributor guide
Assessment
This issue has not been assessed yet.