llvm / llvm/llvm-project

SLP costs broadcast shuffles that are folded

Open
#212,413 10 comments 1 reaction 1 assignee Claimed by @bababuck View on GitHub
backend:RISC-V llvm:SLPVectorizer
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

RISC-V has `vfoo.vx` instructions , which splat a scalar in the second operand. The SLP cost model doesn't account for this though and we end up overcosting by an insertelement + broadcast shuffle:

```llvm
; RUN: opt -p slp-vectorizer -S -o - scratch/slp-splat-cost.ll -mtriple riscv64 -mattr=+v -debug 2<&1 | FileCheck %s

define void @fmul_splat(i32 %s, ptr %p, ptr %q) {
%x0 = load i32, ptr %p
%p1 = getelementptr inbounds nuw i32, ptr %p, i64 1
%x1 = load i32, ptr %p1
%p2 = getelementptr inbounds nuw i32, ptr %p, i64 2
%x2 = load i32, ptr %p2
%p3 = getelementptr inbounds nuw i32, ptr %p, i64 3
%x3 = load i32, ptr %p3
%m0 = add i32 %s, %x0
%m1 = add i32 %s, %x1
%m2 = add i32 %s, %x2
%m3 = add i32 %s, %x3
store i32 %m0, ptr %q
%q1 = getelementptr i32, ptr %q, i64 1
store i32 %m1, ptr %q1
%q2 = getelementptr i32, ptr %q, i64 2
store i32 %m2, ptr %q2
%q3 = getelementptr i32, ptr %q, i64 3
store i32 %m3, ptr %q3
ret void
}
```

```
SLP: Calculated costs for Tree:
1.
Operand 0:
i32 %s
i32 %s
i32 %s
i32 %s
Operand 1:
%x0 = load i32, ptr %p, align 4
%x1 = load i32, ptr %p1, align 4
%x2 = load i32, ptr %p2, align 4
%x3 = load i32, ptr %p3, align 4
Scalars:
%m0 = add i32 %s, %x0
%m1 = add i32 %s, %x1
%m2 = add i32 %s, %x2
%m3 = add i32 %s, %x3
State: Vectorize
MainOp: %m0 = add i32 %s, %x0
AltOp: %m0 = add i32 %s, %x0
VectorizedValue: NULL
ReuseShuffleIndices: Empty
ReorderIndices:
UserTreeIndex: {User:0 EdgeIdx:0}
SLP: Costs:
SLP: ReuseShuffleCost = 0
SLP: VectorCost = 1
SLP: ScalarCost = 4
SLP: ReuseShuffleCost + VecCost - ScalarCost = -3
SLP: Adding cost -3 for bundle Idx: 1, n=4 [ %m0 = add i32 %s, %x0, ..].
SLP: Current total cost = -6
SLP: Adding cost 2 for bundle Idx: 2, n=4 [i32 %s, ..]. <------ # THIS SHOULD BE FREE
SLP: Current total cost = -4
```

I believe this overcosting is contributing to the performance regressions we're seeing with SLP whenever it's moved after unrolling, which causes it to stop vectorizing, see https://github.com/llvm/llvm-project/pull/135318#issuecomment-5099702573

I'm not sure what the correct TTI API should look like. We have `canSplatOperand(unsigned Opcode, int Operand)` in RISCVTTIImpl but it's not a generic TTI hook.

cc @bababuck @alexey-bataev

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.