[LoopVectorize] Missed vectorization when a reference is wrapped in a by-value aggregate
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Consider this:
```cpp
#include
constexpr int size = 1024;
struct View {
const std::vector& values;
int operator[](int i) const { return values[i]; }
};
void direct(int* __restrict flags, const std::vector& values,
const int* __restrict nulls) {
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && values[i];
}
void reference(int* __restrict flags, const std::vector& values,
const int* __restrict nulls) {
const auto& ref = values;
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && ref[i];
}
void view(int* __restrict flags, View values,
const int* __restrict nulls) {
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && values[i];
}
```
Godbolt:
https://godbolt.org/z/e514TbYv6
Clang trunk, `-O3 -march=x86-64-v3 -std=c++20`, vectorizes `direct` and
`reference` with vectorization width 8 and interleave count 4, but does not
vectorize `view`:
```text
remark: the cost-model indicates that vectorization is not beneficial
remark: the cost-model indicates that interleaving is not beneficial
```
The only relevant difference is that `view` wraps the same
`const std::vector&` in a small aggregate passed by value. The local
reference in `reference` does not inhibit vectorization.
For a direct reference parameter, Clang emits attributes similar to:
```llvm
ptr nonnull readonly align 8 dereferenceable(24) %values
```
and loads the vector's data pointer once before the vectorized loop.
For the by-value `View` parameter, the reference member is ABI-lowered to a
plain pointer argument without the same `nonnull` / `dereferenceable`
information. In the optimized scalar IR, loading the vector's data pointer
remains inside the short-circuit branch, conceptually:
```llvm
if (!nulls[i]) {
%data = load ptr, ptr %view_argument
%value = load i32, ptr %data[i]
}
```
Changing `&&` to non-short-circuit `&`, or explicitly loading the data pointer
before the loop, allows the loop to vectorize:
```cpp
const int* data = values.values.data();
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && data[i];
```
Is this a missed optimization caused by reference validity information being
lost when the reference is wrapped in a by-value aggregate? It seems that the
data-pointer load should be safe to hoist in this case, allowing `view` to be
vectorized like `direct`.
Contributor guide
Assessment
This issue has not been assessed yet.