llvm / llvm/llvm-project

[X86][LoopVectorize] Autovectorization leaves dead no-exit vector loop for periodic load

Open
#196,726 8 comments 1 reaction 0 assignees View on GitHub
backend:X86 missed-optimization vectorizers
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

A single scalar loop with a periodic load from an 8-element bias table can produce an unreachable vector loop that has no exit edge:

```asm
.LBB0_7:
...
add rax, 16
jmp .LBB0_7
```

In the local Clang 22.1.4 output this block appears to be dead, not an actually executable infinite loop. However, it is emitted in both Linux SysV and Win64 builds. On Win64 the same dead block also affects register allocation: it uses XMM6/XMM7, so the function gets nonvolatile XMM prologue spills and epilogue reloads.

This is separate from the smaller cost-model issue around manually vectorized loops with scalar tails. The one-loop source below lets LLVM own the vectorization plan, but LoopVectorize still leaves behind a dead no-exit vector loop. The generated function is larger than necessary on Linux and Win64; Win64 additionally pays ABI save/restore cost for registers only used by the dead block.

## Reproducer

This is header-free so it can be pasted directly into [Compiler Explorer](https://gcc.godbolt.org/z/r9Wh3sbW3)(dead code at end) when targeting Win64 from a Linux-hosted Clang. It avoids `` and ``, which may require a target sysroot.

```c
typedef unsigned char u8;

__attribute__((noinline, target("avx")))
void repro(const short *src, u8 *dst, int n, const short *bias) {
for (int i = 0; i < n; ++i) {
int v = (src[i] + bias[i & 7]) >> 7;
if (v < 0) v = 0;
if (v > 255) v = 255;
dst[i] = (u8)v;
}
}
```

Compile with either of these:

```bash
-O3 -target x86_64-pc-windows-msvc -fno-stack-protector -masm=intel
-O3 -target x86_64-unknown-linux-gnu -fno-stack-protector -masm=intel
```

The Win64 output starts by saving XMM6/XMM7:

```asm
repro:
push rsi
push rdi
push rbp
push rbx
sub rsp, 40
vmovdqa xmmword ptr [rsp + 16], xmm7
.seh_savexmm xmm7, 16
vmovdqa xmmword ptr [rsp], xmm6
.seh_savexmm xmm6, 0
```

Later in the function, the unreachable no-exit vector loop remains. The same dead block is also present when compiling for `x86_64-unknown-linux-gnu`; only the register names and calling-convention details differ.

```asm
.LBB0_6:
vpmovsxwd xmm0, qword ptr [r9]
vpmovsxwd xmm1, qword ptr [r9 + 8]
vpmovsxwd xmm2, qword ptr [r9 + 16]
vpmovsxwd xmm3, qword ptr [r9 + 24]
mov eax, 12
.p2align 4
.LBB0_7:
vpmovsxwd xmm4, qword ptr [rcx + 2*rax - 24]
vpaddd xmm4, xmm0, xmm4
vpmovsxwd xmm5, qword ptr [rcx + 2*rax - 16]
vpaddd xmm5, xmm1, xmm5
vpmovsxwd xmm6, qword ptr [rcx + 2*rax - 8]
vpaddd xmm6, xmm2, xmm6
vpmovsxwd xmm7, qword ptr [rcx + 2*rax]
vpaddd xmm7, xmm3, xmm7
vpsrad xmm4, xmm4, 7
vpackssdw xmm4, xmm4, xmm4
vpsrad xmm5, xmm5, 7
vpackssdw xmm5, xmm5, xmm5
vpackuswb xmm4, xmm4, xmm5
vpsrad xmm5, xmm6, 7
vpackssdw xmm5, xmm5, xmm5
vpsrad xmm6, xmm7, 7
vpackssdw xmm6, xmm6, xmm6
vpackuswb xmm5, xmm5, xmm6
vshufps xmm4, xmm4, xmm5, 136
vmovups xmmword ptr [rdx + rax - 12], xmm4
add rax, 16
jmp .LBB0_7
```

## Why this appears dead, not executable

In the local output, the only path into the vector version goes through:

```asm
test r8d, r8d
jle .Lreturn
lea r10d, [r8 - 9]
cmp r10d, -5
jae .LBB0_3
```

For positive signed `n`, the unsigned `jae` is true only for `4 <= n <= 8`.
Inside `.LBB0_3`, the branch to the no-exit vector loop is guarded by:

```asm
cmp r8d, 16
jae .LBB0_6
```

That condition is impossible on the only reachable range for `.LBB0_3`. So the vector loop is dead in this local output, but it is still emitted and still affects register allocation.

## Expected result

LLVM should eliminate the dead vector loop before final code emission. For Win64 specifically, the dead block also must not influence callee-saved register decisions: the output should not save XMM6/XMM7 when those registers are only used in an unreachable block.

Disabling loop vectorization removes the no-exit vector block on both SysV and Win64, and removes the Win64 XMM saves:

```bash
-O3 -target x86_64-pc-windows-msvc -fno-stack-protector -masm=intel -fno-vectorize
```

The loop pragma also removes the symptom:

```c
#pragma clang loop vectorize(disable)
for (int i = 0; i < n; ++i) {
...
}
```

Contributor guide

Open the contributing guide

Research direction

Start with the header-free C reproducer in Compiler Explorer and compile it with the two provided x86-64 Clang commands, comparing the emitted SysV and Win64 assembly. Trace the LoopVectorize-generated path leading to .LBB0_6/.LBB0_7 and verify the reachability conditions. Done means the unreachable no-exit vector loop is absent and Win64 no longer saves XMM6/XMM7 solely for that block.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.