llvm / llvm/llvm-project

[LoopVectorize] Missing vectorization for conditional self-assignment

Open
#212,787 6 comments 4 reactions 2 assignees Claimed by @VachanVY View on GitHub
missed-optimization vectorizers
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Consider this:

```cpp
#include
#include

extern "C" __attribute__((noinline)) void scalar_select(
std::uint64_t* __restrict result,
const std::uint8_t* __restrict condition,
const std::uint64_t* __restrict input,
std::size_t rows_count) {
for (std::size_t row = 0; row < rows_count; ++row) {
result[row] = condition[row] ? input[row] : result[row];
}
}
```

Godbolt:

https://godbolt.org/z/jYrc4qanj

Clang trunk, `-std=c++20 -O3 -march=x86-64-v3`, does not vectorize `scalar_select`:

```text
loop not vectorized: unsafe dependent memory operations in loop
Unsafe indirect dependence.
```

GCC trunk vectorizes the same loop. Clang also vectorizes the equivalent conditional-update form included in the Godbolt link:

```cpp
if (condition[row]) {
result[row] = input[row];
}
```

All three pointers are `__restrict`, so aliasing between the arrays is excluded.

Clang lowers the conditional lvalue to a pointer select followed by a load:

```llvm
%base = select i1 %condition, ptr %input, ptr %result
%ptr = getelementptr i64, ptr %base, i64 %row
%value = load i64, ptr %ptr
```

Both selected addresses still use the same `row`, so iteration `N` can only read `input[N]` or `result[N]`. The loop should be vectorizable as a masked conditional update, but LoopAccessAnalysis reports an unsafe indirect dependence.

This is related to #33951, which also involved a load through a pointer select, but that case had an already redundant load that could be eliminated by EarlyCSE. That does not apply to this loop.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.