llvm / llvm/llvm-project

[AArch64][LV]Missed vectorization: early-exit legality check rejects loop that GCC SLP-vectorizes successfully

Open
#194,233 2 comments 0 reactions 0 assignees View on GitHub
backend:AArch64 missed-optimization vectorizers
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

**Description:**

Clang fails to auto-vectorize an inner loop containing an early exit (`break`) on AArch64 with SVE enabled, while GCC successfully applies SLP vectorization to the same loop under identical compilation flags.

The inner loop contains a conditional `break` when `a[idx] > c[idx]`, and computes `idx = j * m + i` with `idx * 17` strided accesses to multiple `__restrict__` arrays. Clang 21.1.1 reports that it cannot vectorize early exit loops. GCC 16.0.1, while also rejecting full loop vectorization, successfully applies SLP vectorization to the basic block containing the two loads and addition. This demonstrates that Clang's early-exit check precludes SLP vectorization of the loop body, while GCC's SLP operates independently of the loop vectorizer's early-exit decision.

**Test case:**
```c
#include
#include

char foo(
const int * __restrict__ a,
const int * __restrict__ b,
const int * __restrict__ c,
unsigned int * __restrict__ out,
int n,
int m) {
for (int j = m - 1; j >= 0; j -= 4)
{
for (int i = n - 1; i >= 0; i -= 1)
{
int idx = j * m + i;
out[idx] = (((unsigned int)a[(idx * 17)]) + ((unsigned int)b[(idx * 17)]));
if ((a[idx] > c[idx])) {
break;
}
}
}
return (char)0;
}
```

**clang version (Clang 21.1.1):**
```
clang version 21.1.1
Target: unknown
Thread model: posix
Build config: +unoptimized, +assertions
```

**Clang options:**
```
-S -O3 -ftree-vectorize -ftree-slp-vectorize --target=aarch64-linux-gnu -march=armv9-a+sve -Rpass=.*vectorize.* -Rpass-missed=.*vectorize.* -Rpass-analysis=.*vectorize.*
```

**The result of Clang 21.1.1:**
```
test.c:13:9: remark: loop not vectorized: Cannot vectorize early exit loop [-Rpass-analysis=loop-vectorize]
13 | for (int i = n - 1; i >= 0; i -= 1)
| ^
test.c:13:9: remark: loop not vectorized [-Rpass-missed=loop-vectorize]
test.c:16:72: remark: Cannot SLP vectorize list: vectorization was impossible with available vectorization factors [-Rpass-missed=slp-vectorizer]
16 | out[idx] = (((unsigned int)a[(idx * 17)]) + ((unsigned int)b[(idx * 17)]));
| ^
test.c:17:18: remark: Cannot SLP vectorize list: vectorization was impossible with available vectorization factors [-Rpass-missed=slp-vectorizer]
17 | if ((a[idx] > c[idx])) {
| ^
test.c:16:40: remark: Cannot SLP vectorize list: vectorization was impossible with available vectorization factors [-Rpass-missed=slp-vectorizer]
16 | out[idx] = (((unsigned int)a[(idx * 17)]) + ((unsigned int)b[(idx * 17)]));
| ^
```

Also reproducible on Godbolt (https://godbolt.org/z/5Pz4zKWMh).

**Preliminary root cause analysis:**

Through instrumentation of Clang's loop vectorization pipeline, I have narrowed down the failure to the **early-exit legality checks** stage.

The `break` statement causes the loop to have no single exiting block, triggering early-exit legality checks which determine that the loop cannot be safely vectorized. This rejection prevents any vectorization factor (VF) from being established, which in turn causes the SLP vectorizer to report `vectorization was impossible with available vectorization factors`.

**However, GCC vectorizes the loop body using SLP.** GCC version 16.0.1.

**gcc version:**
```
aarch64-unknown-linux-gnu-gcc (GCC) 16.0.1 20260413 (experimental)
Copyright (C) 2026 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```

**GCC options:**
```
-S -march=armv9-a+sve -ftree-vectorize -O3 -fopt-info-vec-all -fvect-cost-model=unlimited
```

**The result of GCC:**
```
test.c:11:27: missed: couldn't vectorize loop
test.c:11:27: missed: not vectorized: unsupported outerloop form.
test.c:13:31: missed: couldn't vectorize loop
test.c:13:31: missed: unsupported SLP instances
test.c:4:6: note: vectorized 0 loops in function.
test.c:16:55: note: ***** Analysis failed with vector mode VNx4SI
test.c:16:55: note: ***** The result for vector mode VNx16QI would be the same
test.c:16:55: note: ***** The result for vector mode VNx8QI would be the same
test.c:16:55: note: ***** The result for vector mode VNx4QI would be the same
test.c:16:55: note: ***** Re-trying analysis with vector mode VNx2QI
test.c:16:55: note: ***** Analysis failed with vector mode VNx2QI
test.c:16:55: note: ***** Re-trying analysis with vector mode V16QI
test.c:16:55: note: ***** Analysis succeeded with vector mode V16QI
test.c:16:55: note: SLPing BB part
test.c:16:55: note: Basic block will be vectorized using SLP
test.c:16:55: optimized: basic block part vectorized using 8 byte vectors
test.c:16:55: note: Vectorizing SLP tree:
test.c:16:55: note: Root stmt: _14 = _7 + _10;
test.c:16:55: note: node (max_nunits=2, refcnt=1) vector(2) unsigned int
test.c:16:55: note: op template: _7 = (unsigned int) _6;
test.c:16:55: note: stmt 0 _7 = (unsigned int) _6;
test.c:16:55: note: stmt 1 _10 = (unsigned int) _9;
test.c:16:55: note: children (max_nunits=1, refcnt=1) vector(2) int
test.c:16:55: note: { _6, _9 }
test.c:16:55: note: ------>vectorizing SLP node starting from: _7 = (unsigned int) _6;
test.c:16:55: note: transform assignment.
test.c:16:55: note: add new stmt: vect__7.5_59 = VIEW_CONVERT_EXPR(_58);
test.c:16:55: note: vectorizing stmts using SLP.
```

Also reproducible on Godbolt: https://godbolt.org/z/oz9er9aPG

**GCC Assembly (SLP vectorized inner loop body):**
```asm
.L4:
ldr s31, [x8, x1] ; load a[idx*17] → s31 (32-bit lane)
ldr s30, [x7, x1] ; load b[idx*17] → s30
sub x1, x1, #68
ldr w3, [x4, x0] ; load a[idx] for break condition
ldr w2, [x5, x0] ; load c[idx] for break condition
uzp1 v31.2s, v31.2s, v30.2s ; pack a and b into v31.2s
addp v31.2s, v31.2s, v31.2s ; horizontal add → a + b
str s31, [x6, x0] ; store to out[idx]
sub x0, x0, #4
cmp w3, w2 ; break condition
ble .L9
```

**Analysis:**

1. **Primary Issue — Early-exit legality check blocks all vectorization:** The `break` statement causes Clang's loop vectorizer to reject the inner loop at the early-exit legality checks stage. This prevents any vectorization factor (VF) from being established.

2. **Secondary Issue — SLP cascade failure:** Without a VF from the loop vectorizer, the SLP vectorizer reports `vectorization was impossible with available vectorization factors`. The SLP failures are a downstream consequence of the loop vectorizer's early-exit rejection.

3. **GCC's alternative approach — SLP independent of loop vectorization:** GCC also rejects full loop vectorization (`missed: couldn't vectorize loop`), but its SLP vectorizer independently analyzes the basic block and successfully packs `a[idx*17]` and `b[idx*17]` using `V16QI` (8-byte vectors, 2×32-bit lanes), generating `uzp1` + `addp` for efficient horizontal addition.

The affected pattern—loops with early exits that perform multiple independent loads and arithmetic—appears in real-world code processing multi-dimensional arrays with early termination conditions. GCC demonstrates that SLP vectorization of the loop body is achievable and beneficial even when full loop vectorization is impossible.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.