[SelectOptimize] Perf regression: clang 10+ converts predictable branches to cmov chains; SelectOptimize disabled by default
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
# Perf regression: clang 10+ converts predictable branches to cmov chains (SelectOptimize disabled by default)
## Summary
Since clang 10, `SimplifyCFG` speculatively converts conditional branches to
`select` (cmov) instructions even when those branches are highly predictable.
`SelectOptimize`, the pass designed to revert this when unprofitable, is
disabled by default, so the cmov chains are never rolled back.
For code with tight loops and predictable branches (e.g., digit-string parsing),
this raises instruction count by ~49% and cycle count by ~49% vs clang 9.
## Reproducer
```cpp
// mre_unroll2_clang10.cc
// Compile: clang++ -O2 -std=c++17 -S -o bad.s mre_unroll2_clang10.cc
// Check: grep -c 'cmov' bad.s # 81 on clang 18, 0 on clang 9
#include
#include
using pair32 = std::pair;
template
struct calc {
std::pair
operator()(const char* s1, const char* s2, uint32_t n1, uint32_t n2) {
if (*s1 & *s2) {
n1 = n1 * 10 + (uint8_t)(*s1++) - '0';
n2 = n2 * 10 + (uint8_t)(*s2++) - '0';
return calc()(s1, s2, n1, n2);
} else if (*s1) {
for (int i = 0; i < N; i++)
if (*s1) { n1 = n1 * 10 + (uint8_t)(*s1++) - '0'; }
return {n1, n2};
} else {
for (int i = 0; i < N; i++)
if (*s2) { n2 = n2 * 10 + (uint8_t)(*s2++) - '0'; }
return {n1, n2};
}
}
};
template<> struct calc<0> {
std::pair
operator()(const char*, const char*, uint32_t n1, uint32_t n2) {
return {n1, n2};
}
};
__attribute__((noinline))
std::pair convert_pair(const char* s1, const char* s2) {
uint32_t n1 = (uint8_t)(*s1++) - '0';
uint32_t n2 = (uint8_t)(*s2++) - '0';
return calc<9>()(s1, s2, n1, n2);
}
volatile const char* g_s1;
volatile const char* g_s2;
int main() {
g_s1 = "1234"; g_s2 = "5678";
auto [a, b] = convert_pair((const char*)g_s1, (const char*)g_s2);
return (int)(a + b);
}
```
## How to reproduce
```bash
clang++-9 -O2 -std=c++17 -S -o good.s mre_unroll2_clang10.cc && grep -c cmov good.s # 0
clang++-10 -O2 -std=c++17 -S -o bad.s mre_unroll2_clang10.cc && grep -c cmov bad.s # 81
clang++-19 -O2 -std=c++17 -S -o bad.s mre_unroll2_clang10.cc && grep -c cmov bad.s # 81
```
## Impact
Measured on AMD Ryzen 9 5950X with 500 iterations over 41K digit strings:
| Compiler | Cycles | Instructions | IPC | cmov count |
|------------|---------|--------------|------|------------|
| clang++ 9 | 14.5 B | 49.8 B | 3.43 | 0 |
| clang++ 10 | 21.7 B | 73.8 B | 3.39 | 81 |
| clang++ 18 | 21.7 B | 73.8 B | 3.39 | 81 |
**+49% cycle regression**, present in every clang release since 10.
## Root cause
Three interacting issues:
### 1. `SelectOptimize` is disabled by default (`TargetPassConfig.cpp`)
```cpp
// llvm/lib/CodeGen/TargetPassConfig.cpp ~line 262
static cl::opt DisableSelectOptimize(
"disable-select-optimize", cl::init(true), // always disabled
...);
```
`SelectOptimize` was added precisely to revert unprofitable select-to-branch
lowering done by `SimplifyCFG`. With it off, the pass never runs.
### 2. `SelectOptimize` has no no-PGO heuristic for dense selects
When profile data is unavailable (the common case), `isConvertToBranchProfitableBase()` returns `false` for non-loop BBs unless branch weights or cold-operand sinking apply. The digit-string BBs have neither, so even if SelectOptimize ran, it would leave the cmov chains intact.
### 3. `CodeGenPrepare` and `X86CmovConversion` bail out when `SelectOptimize` is enabled
Both passes contain:
```cpp
if (!getCGPassBuilderOption().DisableSelectOptimize)
return false;
```
This makes them mutually exclusive with `SelectOptimize`. Since the passes operate at different IR/machine levels and are naturally complementary (a select converted by SelectOptimize simply disappears before the later pass sees it), the bail-outs are unnecessary and prevent the later passes from handling machine-level cmovs that SelectOptimize never touched.
## Suggested fix
See the attached patch. In summary:
1. Enable `SelectOptimize` by default (`cl::init(false)` for `DisableSelectOptimize`).
2. Add a density heuristic to `SelectOptimize`: when no PGO data is present and the target considers predictable selects expensive (out-of-order x86), revert BBs with >= 4 scalar selects to branches. This threshold is tunable via `-select-opti-nopgo-density-threshold`.
3. Fix the `!unpredictable` metadata check in `findProfitableSIGroupsInnerLoops`: the base path checked for `MD_unpredictable`, but the loop-analysis path did not; add the same guard.
4. Remove the early-return bail-outs from `CodeGenPrepare::optimizeSelectInst` and `X86CmovConversion`, replacing them with an explanatory comment.
With this fix, the reproducer compiles to 6 cmov instructions (down from 81).
## Affected versions
Clang 10 through 18 (and likely current trunk).
## Compiler Explorer
[https://godbolt.org//#eyJzZXNzaW9ucyI6W3siaWQiOjEsImxhbmd1YWdlIjoiYysrIiwic291cmNlIjoiLy8gbXJlX3Vucm9sbDJfY2xhbmcxMC5jY1xyXG4vLyBDb21waWxlOiBjbGFuZysrIC1PMiAtc3RkPWMrKzE3IC1TIC1vIGJhZC5zIG1yZV91bnJvbGwyX2NsYW5nMTAuY2NcclxuLy8gQ2hlY2s6ICAgZ3JlcCAtYyAnY21vdicgYmFkLnMgICAjIDgxIG9uIGNsYW5nIDE4LCAwIG9uIGNsYW5nIDlcclxuXHJcbiNpbmNsdWRlIDxjc3RkaW50PlxyXG4jaW5jbHVkZSA8dXRpbGl0eT5cclxuXHJcbnVzaW5nIHBhaXIzMiA9IHN0ZDo6cGFpcjx1aW50MzJfdCwgdWludDMyX3Q-O1xyXG5cclxudGVtcGxhdGU8aW50IE4-XHJcbnN0cnVjdCBjYWxjIHtcclxuICAgIHN0ZDo6cGFpcjx1aW50MzJfdCx1aW50MzJfdD5cclxuICAgIG9wZXJhdG9yKCkoY29uc3QgY2hhciogczEsIGNvbnN0IGNoYXIqIHMyLCB1aW50MzJfdCBuMSwgdWludDMyX3QgbjIpIHtcclxuICAgICAgICBpZiAoKnMxICYgKnMyKSB7XHJcbiAgICAgICAgICAgIG4xID0gbjEgKiAxMCArICh1aW50OF90KSgqczErKykgLSAnMCc7XHJcbiAgICAgICAgICAgIG4yID0gbjIgKiAxMCArICh1aW50OF90KSgqczIrKykgLSAnMCc7XHJcbiAgICAgICAgICAgIHJldHVybiBjYWxjPE4tMT4oKShzMSwgczIsIG4xLCBuMik7XHJcbiAgICAgICAgfSBlbHNlIGlmICgqczEpIHtcclxuICAgICAgICAgICAgZm9yIChpbnQgaSA9IDA7IGkgPCBOOyBpKyspXHJcbiAgICAgICAgICAgICAgICBpZiAoKnMxKSB7IG4xID0gbjEgKiAxMCArICh1aW50OF90KSgqczErKykgLSAnMCc7IH1cclxuICAgICAgICAgICAgcmV0dXJuIHtuMSwgbjJ9O1xyXG4gICAgICAgIH0gZWxzZSB7XHJcbiAgICAgICAgICAgIGZvciAoaW50IGkgPSAwOyBpIDwgTjsgaSsrKVxyXG4gICAgICAgICAgICAgICAgaWYgKCpzMikgeyBuMiA9IG4yICogMTAgKyAodWludDhfdCkoKnMyKyspIC0gJzAnOyB9XHJcbiAgICAgICAgICAgIHJldHVybiB7bjEsIG4yfTtcclxuICAgICAgICB9XHJcbiAgICB9XHJcbn07XHJcbnRlbXBsYXRlPD4gc3RydWN0IGNhbGM8MD4ge1xyXG4gICAgc3RkOjpwYWlyPHVpbnQzMl90LHVpbnQzMl90PlxyXG4gICAgb3BlcmF0b3IoKShjb25zdCBjaGFyKiwgY29uc3QgY2hhciosIHVpbnQzMl90IG4xLCB1aW50MzJfdCBuMikge1xyXG4gICAgICAgIHJldHVybiB7bjEsIG4yfTtcclxuICAgIH1cclxufTtcclxuXHJcbl9fYXR0cmlidXRlX18oKG5vaW5saW5lKSlcclxuc3RkOjpwYWlyPHVpbnQzMl90LHVpbnQzMl90PiBjb252ZXJ0X3BhaXIoY29uc3QgY2hhciogczEsIGNvbnN0IGNoYXIqIHMyKSB7XHJcbiAgICB1aW50MzJfdCBuMSA9ICh1aW50OF90KSgqczErKykgLSAnMCc7XHJcbiAgICB1aW50MzJfdCBuMiA9ICh1aW50OF90KSgqczIrKykgLSAnMCc7XHJcbiAgICByZXR1cm4gY2FsYzw5PigpKHMxLCBzMiwgbjEsIG4yKTtcclxufVxyXG5cclxudm9sYXRpbGUgY29uc3QgY2hhciogZ19zMTtcclxudm9sYXRpbGUgY29uc3QgY2hhciogZ19zMjtcclxuXHJcbmludCBtYWluKCkge1xyXG4gICAgZ19zMSA9IFwiMTIzNFwiOyBnX3MyID0gXCI1Njc4XCI7XHJcbiAgICBhdXRvIFthLCBiXSA9IGNvbnZlcnRfcGFpcigoY29uc3QgY2hhciopZ19zMSwgKGNvbnN0IGNoYXIqKWdfczIpO1xyXG4gICAgcmV0dXJuIChpbnQpKGEgKyBiKTtcclxufSIsImNvbmZvcm1hbmNldmlldyI6ZmFsc2UsImNvbXBpbGVycyI6W3siX2ludGVybmFsaWQiOjEsImlkIjoiY2xhbmcxOTEwIiwib3B0aW9ucyI6Ii1PMiAtc3RkPWMrKzE3IiwiZmlsdGVycyI6eyJiaW5hcnkiOmZhbHNlLCJiaW5hcnlPYmplY3QiOmZhbHNlLCJjb21tZW50T25seSI6dHJ1ZSwiZGVtYW5nbGUiOnRydWUsImRpcmVjdGl2ZXMiOnRydWUsImV4ZWN1dGUiOmZhbHNlLCJpbnRlbCI6dHJ1ZSwibGFiZWxzIjp0cnVlLCJsaWJyYXJ5Q29kZSI6ZmFsc2UsInRyaW0iOmZhbHNlLCJkZWJ1Z0NhbGxzIjpmYWxzZX0sImxpYnMiOltdLCJzcGVjaWFsb3V0cHV0cyI6W10sInRvb2xzIjpbXSwib3ZlcnJpZGVzIjpbXX1dLCJleGVjdXRvcnMiOltdfV0sInRyZWVzIjpbXX0](link)
Contributor guide
Assessment
This issue has not been assessed yet.