[AArch64] Loop Vectorizer fails to form interleaved store group (st2) when pointers are outer-loop inductions
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
## Summary
The Loop Vectorizer fails to form a factor-2 interleaved store group when two stride-2 store pointers are carried across an outer loop. The cost model then sees the stores as expensive scalar scatters and rejects vectorization entirely as "not beneficial."
The same logical store pattern in a single (non-nested) loop correctly forms the interleave group, vectorizes, and generates `st2`.
## Reproduction
```cpp
// WORKS: vectorized with st2
void single_loop(const float* __restrict src, float* __restrict dst, unsigned n) {
float* pA = dst;
float* pB = dst + 1;
for (unsigned i = 0; i < n; i++) {
*pA = src[i] * 2.0f;
*pB = src[i] * 3.0f;
pA += 2;
pB += 2;
}
}
// FAILS: not vectorized ("not beneficial")
void nested_loop(const float* __restrict src, float* __restrict dst,
unsigned width, unsigned height, unsigned srcStride) {
float* pA = dst;
float* pB = dst + 1;
for (unsigned y = 0; y < height; y++) {
const float* row = src + y * srcStride;
for (unsigned x = 0; x < width; x++) {
*pA = row[x] * 2.0f;
*pB = row[x] * 3.0f;
pA += 2;
pB += 2;
}
}
}
```
Compile with:
```
clang --target=aarch64-linux-gnu -O2 -S test.cpp -Rpass=loop-vectorize -Rpass-missed=loop-vectorize
```
## Observed Behavior
`single_loop`: vectorized (width 4, interleaved count 2), generates optimal `st2`:
```asm
st2 { v5.4s, v6.4s }, [x11]
st2 { v3.4s, v4.4s }, [x14]
```
`nested_loop`: **not vectorized at all**:
```
remark: the cost-model indicates that vectorization is not beneficial
remark: the cost-model indicates that interleaving is not beneficial
```
If vectorization is forced with `#pragma clang loop vectorize(enable) vectorize_width(4)`, the computation is vectorized but stores are emitted as scalar lane extracts:
```asm
st1 { v3.s }[1], [x7]
st1 { v3.s }[3], [x7]
st1 { v3.s }[2], [x1]
st1 { v2.s }[1], [x7]
st1 { v2.s }[2], [x5]
st1 { v2.s }[3], [x7]
```
## Expected Behavior
`nested_loop` should vectorize and generate `st2` instructions, since the inner loop store pattern is identical to `single_loop` (two pointers offset by 4 bytes, both advancing by stride 8 bytes per iteration).
## Impact
This pattern is common in image processing code that computes two (or more) output channels stored interleaved in memory, with a 2D loop nest (rows x columns).
The performance impact is significant: `st2` writes 32 bytes in one instruction vs 6+ `st1` lane stores for the same data.
Contributor guide
Research direction
Start with the test.cpp reproduction and run the provided AArch64 clang command, comparing vectorizer remarks and generated assembly for the single and nested loops. Trace the loop-vectorizer handling of the outer-loop induction pointers; done means nested_loop vectorizes without forcing and produces st2 stores rather than scalar lane stores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100