llvm / llvm/llvm-project

[SelectionDAG] Estimate refinement creates illegal FP constants

Open
#217,806 1 comment 0 reactions 1 assignee Claimed by @MattPD View on GitHub
backend:X86 confirmed crash-on-valid
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

DAGCombiner calls the `getSqrtEstimate` and `getRecipEstimate` target lowering hooks for a fast estimate and a refinement step count. A hook can replace an unspecified count with its target default. DAGCombiner then builds the requested Newton-Raphson refinement.

The refinement builds floating-point constants of the estimate's element type. For a vector estimate, that element type can be illegal even when the vector type is legal.

The failure occurs on 32-bit X86 with this feature configuration: SSE enabled, SSE2 disabled, and x87 disabled. `v4f32` is legal in that configuration, but scalar `f32` is illegal. `X86TargetLowering::getSqrtEstimate` can therefore return a legal `v4f32` reciprocal square root estimate. `DAGCombiner::buildSqrtNRTwoConst` then builds a `BUILD_VECTOR` that splats `f32` constants. Type legalization aborts on that node, so valid IR crashes the backend.

Compiler Explorer reproducer: https://godbolt.org/z/fE36zoaYG

```llvm
target triple = "i686-unknown-linux-gnu"

declare <4 x float> @llvm.sqrt.v4f32(<4 x float>)

define <4 x float> @rsqrt_v4f32(<4 x float> %n, <4 x float> %x) #0 {
%sqrt = call afn ninf <4 x float> @llvm.sqrt.v4f32(<4 x float> %x)
%q = fdiv arcp ninf <4 x float> %n, %sqrt
ret <4 x float> %q
}

attributes #0 = {
"reciprocal-estimates"="vec-sqrtf"
"target-features"="+sse,-sse2,-x87"
}
```

```text
llc -O3 repro.ll -o /dev/null
```

The backend reports the following, with surrounding output elided:

```text
SoftenFloatOperand ... v4f32 = BUILD_VECTOR ... ConstantFP:f32<-5.000000e-01>
fatal error: error in backend: Do not know how to soften this operator's operand!
```

Reciprocal refinement fails the same way above one refinement step. `DAGCombiner::BuildDivEstimate` builds a vector `1.0` constant. With one step, the final iteration uses the numerator instead, so the constant has no remaining uses and the case still compiles. With two or more steps, the constant remains live and exposes the illegal element type.

Square-root refinement builds constants whenever its refinement step count is positive. Reciprocal refinement needs a live constant only above one step. A fix therefore needs to preserve the currently working boundaries: zero-step square-root estimates, and zero-step or one-step reciprocal estimates.

Issue https://github.com/llvm/llvm-project/issues/217801 tracks the estimate node's own result type, a separate problem.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.