llvm / llvm/llvm-project

[X86][VectorCombine] Merge adjacent scalar stores into vector stores when profitable

Open
#198,075 1 comment 1 reaction 1 assignee Claimed by @ParkHanbum View on GitHub
backend:X86 llvm:vectorcombine
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Description

When storing multiple scalar values into adjacent or contiguous memory locations, it may be profitable to combine them into a vector value and perform a single vector store.

For example, the following IR stores four `float` values into a contiguous 16-byte memory region:

```llvm
define void @src(ptr %base, float %f0, float %f1, float %f2, float %f3) {
entry:
%p8 = getelementptr inbounds i8, ptr %base, i64 8
store float %f0, ptr %p8, align 8

%p12 = getelementptr inbounds i8, ptr %base, i64 12
store float %f1, ptr %p12, align 4

%p16 = getelementptr inbounds i8, ptr %base, i64 16
store float %f2, ptr %p16, align 8

%p20 = getelementptr inbounds i8, ptr %base, i64 20
store float %f3, ptr %p20, align 4

ret void
}
```

These stores cover the range [base + 8, base + 24), so they could be represented as a single <4 x float> store:
```llvm
define void @tgt(ptr %base, float %f0, float %f1, float %f2, float %f3) {
entry:
%p8 = getelementptr inbounds i8, ptr %base, i64 8

%v0 = insertelement <4 x float> poison, float %f0, i32 0
%v1 = insertelement <4 x float> %v0, float %f1, i32 1
%v2 = insertelement <4 x float> %v1, float %f2, i32 2
%v3 = insertelement <4 x float> %v2, float %f3, i32 3

store <4 x float> %v3, ptr %p8, align 8
ret void
}
```

This can reduce the number of store operations and may reduce store-port pressure on some targets.

## Performance observation

On x86_64 with -mcpu=znver4, manually forming the vector store improved the llvm-mca block throughput in my reduced testcase:

Before:
Iterations: 100
Instructions: 400
Total Cycles: 303
Total uOps: 400
Block RThroughput: 3.0

After:
Iterations: 100
Instructions: 400
Total Cycles: 204
Total uOps: 400
Block RThroughput: 1.0

Proof: https://alive2.llvm.org/ce/
Sample: https://godbolt.org/z/4c7zjWEx5

Real field usage: https://github.com/dtcxzyw/llvm-opt-benchmark/blob/83bfde85e2ee64eb7ef7d7532404877b5c48a875/bench/pbrt-v4/optimized/camera.ll#L875

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.