llvm / llvm/llvm-project

[LoopVectorize] Missed vectorization when a reference is wrapped in a by-value aggregate

Open
#215,778 4 comments 0 reactions 1 assignee Claimed by @dretoh View on GitHub
missed-optimization vectorizers
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Consider this:

```cpp
#include

constexpr int size = 1024;

struct View {
const std::vector& values;
int operator[](int i) const { return values[i]; }
};

void direct(int* __restrict flags, const std::vector& values,
const int* __restrict nulls) {
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && values[i];
}

void reference(int* __restrict flags, const std::vector& values,
const int* __restrict nulls) {
const auto& ref = values;
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && ref[i];
}

void view(int* __restrict flags, View values,
const int* __restrict nulls) {
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && values[i];
}
```

Godbolt:

https://godbolt.org/z/e514TbYv6

Clang trunk, `-O3 -march=x86-64-v3 -std=c++20`, vectorizes `direct` and
`reference` with vectorization width 8 and interleave count 4, but does not
vectorize `view`:

```text
remark: the cost-model indicates that vectorization is not beneficial
remark: the cost-model indicates that interleaving is not beneficial
```

The only relevant difference is that `view` wraps the same
`const std::vector&` in a small aggregate passed by value. The local
reference in `reference` does not inhibit vectorization.

For a direct reference parameter, Clang emits attributes similar to:

```llvm
ptr nonnull readonly align 8 dereferenceable(24) %values
```

and loads the vector's data pointer once before the vectorized loop.

For the by-value `View` parameter, the reference member is ABI-lowered to a
plain pointer argument without the same `nonnull` / `dereferenceable`
information. In the optimized scalar IR, loading the vector's data pointer
remains inside the short-circuit branch, conceptually:

```llvm
if (!nulls[i]) {
%data = load ptr, ptr %view_argument
%value = load i32, ptr %data[i]
}
```

Changing `&&` to non-short-circuit `&`, or explicitly loading the data pointer
before the loop, allows the loop to vectorize:

```cpp
const int* data = values.values.data();
for (int i = 0; i < size; ++i)
flags[i] = !nulls[i] && data[i];
```

Is this a missed optimization caused by reference validity information being
lost when the reference is wrapped in a by-value aggregate? It seems that the
data-pointer load should be safe to hoist in this case, allowing `view` to be
vectorized like `direct`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.