llvm / llvm/llvm-project

[InstCombine] Join zero/nonzero edge facts to fold a signed range check

Open
#216,950 1 comment 0 reactions 1 assignee Claimed by @ParkHanbum View on GitHub
llvm:instcombine missed-optimization
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

LLVM's current `default` pipeline retains:

```llvm
%nonnegative = icmp sgt i64 %idx, -1
%below = icmp slt i64 %idx, %len
%ok = and i1 %nonnegative, %below
```

The key blocker is at the merged `check` block, not the source-level
`memset` itself. At `entry`, LLVM forms an edge condition `%len == 0`:

```llvm
%zero = icmp eq i64 %len, 0
br i1 %zero, label %check, label %nonzero
```

The zero predecessor gives `%len = 0`. The nonzero predecessor executes:

```llvm
%bytes = shl nuw i64 %len, 2
call void asm sideeffect "", "r"(i64 noundef %bytes)
br label %check
```

which gives `1 <= %len <= 2^62 - 1` on defined executions. At `check`, those
predecessor facts are not represented by a PHI or a joined range fact. LLVM
therefore does not recognize that the global range is
`0 <= %len <= 2^62 - 1`, and misses:

```llvm
%ok = icmp ult i64 %idx, %len
```

The fold deletes one compare and the boolean `and`; it does not remove the
bounds check or the zero initialization.

## Minimal reproducer

The smallest validated semantic POC keeps only a stable index load, the
zero/nonzero entry split, the nonzero-path `shl nuw` with a poison-observing
sink, and the merged check. The output pointer and `llvm.memset` declaration
are not needed to demonstrate the missed fold.

Source (`2026-08-18-minimal-src.ll`):

```llvm
define i1 @range_after_zero_fill_load(ptr noundef %indices, i64 %len) {
entry:
%idx = load i64, ptr %indices, align 8
%zero = icmp eq i64 %len, 0
br i1 %zero, label %check, label %nonzero

nonzero:
%bytes = shl nuw i64 %len, 2
call void asm sideeffect "", "r"(i64 noundef %bytes)
br label %check

check:
%nonnegative = icmp sgt i64 %idx, -1
%below = icmp slt i64 %idx, %len
%ok = and i1 %nonnegative, %below
ret i1 %ok
}
```

Target (`2026-08-18-minimal-tgt.ll`) is identical through the merge and has:

```llvm
check:
%ok = icmp ult i64 %idx, %len
ret i1 %ok
```

The exact entry split is essential. Removing it leaves no zero-edge fact. The
`shl nuw` sink is also essential: `%len != 0` alone does not prove that an
arbitrary nonzero i64 is signed-nonnegative.

If the `shl nuw` were made unconditional, the branch/merge could be removed,
but that would no longer test this missed edge-fact propagation topology.
Removing the poison-observing shift path is unsound: `%len =
0x8000000000000000` is nonzero but signed-negative, and the unsigned target
can accept an index that the signed source rejects.

An apparently smaller form that deletes the nonzero edge and adds
`llvm.assume(%zero)` is deliberately out of scope. It turns every nonzero
execution into undefined behavior and reduces the check to a trivial
zero-length specialization; it does not exercise the edge-fact join that is
missing in the motivating IR.

## Bit-level legality

For the nonzero predecessor, `shl nuw i64 %len, 2` implies:

```text
0 < len <= floor((2^64 - 1) / 4) = 2^62 - 1
```

Therefore `%len` is nonnegative in signed i64 space. If `%idx` is negative,
its unsigned value is at least `2^63`, so both the signed conjunction and
`idx =s 0`.

The stable memory load is part of the proof boundary. A scalar undef-capable
`%idx` formal is not interchangeable: the earlier relaxed forms fail strict
reverse verification. The `noundef` inline-asm operand preserves observation
of poison from the `nuw` shift; it is a proof sink, not a source replacement.

## Why the pattern occurs in ONNX Runtime

The original `MaxUnpool::Compute` source has a zero fill followed by a guarded
scatter store:

```cpp
auto out = Y->MutableDataAsSpan();
std::fill_n(out.data(), out.size(), 0.f);

for (size_t cur_elem = 0; cur_elem < total_elements; ++cur_elem) {
const int64_t idx = I_data[cur_elem];
if (idx < 0 || idx >= static_cast(out.size())) {
return ORT_MAKE_STATUS(...);
}
out[static_cast(idx)] = X_data[cur_elem];
}
```

See the official [source](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cpu/nn/Unpool.cc#L111-L125).
LLVM lowers the four-byte `std::fill_n` zero fill to a dynamic
`llvm.memset`, producing the byte-count `shl nuw`. The signed bounds check was
introduced by [PR #27432](https://github.com/microsoft/onnxruntime/pull/27432)
and [commit 1f05a4c](https://github.com/microsoft/onnxruntime/commit/1f05a4cedf85304fbf27da48471d0d8b3135c233);
the later [commit a894893](https://github.com/microsoft/onnxruntime/commit/a894893f3cfc73f6518bffdc7c8a410a13d5f6ad)
retained the fill/check structure while using `MutableDataAsSpan()`.

The source explains the provenance, but the minimum LLVM blocker is the
edge-fact join at `check`.

## Validation and cost evidence

The minimal source/target both pass `default,verify`; O3 retains the entry
split, nonzero sink, and source/target predicate distinction. Strict
bidirectional Alive2 reports:

```text
1 correct transformations
0 incorrect transformations
0 failed-to-prove transformations
0 Alive2 errors
```

The minimal two-argument form is a semantic reduction. Its structural proxy
is non-regressing on the requested profiles:

| Profile | Instructions | Text bytes | MCA uOps/100 | Block RThroughput |
| --- | ---: | ---: | ---: | ---: |
| x86-64-v4 | 10 → 7 | 31 → 23 | 12 → 9 | 2.0 → 1.5 |
| Neoverse-V1 | 7 → 6 | 28 → 24 | 7 → 6 | 1.0 → 1.0 |
| RV64 P670 RVV128 | 7 → 5 | 22 → 14 | 7 → 5 | 1.8 → 1.3 |

The earlier ABI-safe inline-asm follow-on remains the canonical predicate-cost
evidence because the two-argument form overlaps an input register with the
boolean return on some targets. Neither measurement is an application-level
ONNX Runtime wall-clock benchmark.

### Cost-sink substitution check (2026-08-18)

The requested literal `call @llvm.use` was checked against the exact LLVM
24.0.0git build used for this packet. It is not a recognized intrinsic; the
verifier reports `unknown intrinsic 'llvm.use'`. An arbitrary declaration with
that reserved name was not used as a substitute. The supported LLVM no-code
use intrinsic is variadic `@llvm.fake.use`, so a cost-only rerun changed only
the sink to:

```llvm
call void (...) @llvm.fake.use(i64 noundef %bytes)
```

`llvm.fake.use` lowers to a `fake_use` marker and emits no machine instruction.
With the same O3 and target profiles, its isolated structural results exactly
match the minimal inline-asm sink baseline:

| Profile | Source → target instructions | Text bytes | MCA uOps/100 | Block RThroughput |
| --- | ---: | ---: | ---: | ---: |
| x86-64-v4 | 10 → 7 | 31 → 23 | 12 → 9 | 2.0 → 1.5 |
| Neoverse-V1 | 7 → 6 | 28 → 24 | 7 → 6 | 1.0 → 1.0 |
| RV64 P670 RVV128 | 7 → 5 | 22 → 14 | 7 → 5 | 1.8 → 1.3 |

## Current LLVM miss and proposed initial scope

The current pipeline appears to lose the predecessor-specific range facts at
the CFG merge. The initial investigation should therefore target only a
path-sensitive range/definedness join for this exact shape:

- one stable loaded integer `%idx`;
- one `%len` used by the entry equality, both signed compares, and the
nonzero-path `shl nuw`;
- zero edge bypasses the shift and nonzero edge cannot bypass it;
- the shifted value is poison-observed by a side-effecting length consumer;
- no `llvm.assume`, `freeze`, range metadata, source-only invariant, or
scalar-undef generalization.

The issue does not prescribe whether the implementation belongs in
InstCombine, ValueTracking, or another middle-end facility; it asks whether
the edge facts can be joined without inventing an assumption.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.