[SelectionDAG] Estimate refinement creates illegal FP constants
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
DAGCombiner calls the `getSqrtEstimate` and `getRecipEstimate` target lowering hooks for a fast estimate and a refinement step count. A hook can replace an unspecified count with its target default. DAGCombiner then builds the requested Newton-Raphson refinement.
The refinement builds floating-point constants of the estimate's element type. For a vector estimate, that element type can be illegal even when the vector type is legal.
The failure occurs on 32-bit X86 with this feature configuration: SSE enabled, SSE2 disabled, and x87 disabled. `v4f32` is legal in that configuration, but scalar `f32` is illegal. `X86TargetLowering::getSqrtEstimate` can therefore return a legal `v4f32` reciprocal square root estimate. `DAGCombiner::buildSqrtNRTwoConst` then builds a `BUILD_VECTOR` that splats `f32` constants. Type legalization aborts on that node, so valid IR crashes the backend.
Compiler Explorer reproducer: https://godbolt.org/z/fE36zoaYG
```llvm
target triple = "i686-unknown-linux-gnu"
declare <4 x float> @llvm.sqrt.v4f32(<4 x float>)
define <4 x float> @rsqrt_v4f32(<4 x float> %n, <4 x float> %x) #0 {
%sqrt = call afn ninf <4 x float> @llvm.sqrt.v4f32(<4 x float> %x)
%q = fdiv arcp ninf <4 x float> %n, %sqrt
ret <4 x float> %q
}
attributes #0 = {
"reciprocal-estimates"="vec-sqrtf"
"target-features"="+sse,-sse2,-x87"
}
```
```text
llc -O3 repro.ll -o /dev/null
```
The backend reports the following, with surrounding output elided:
```text
SoftenFloatOperand ... v4f32 = BUILD_VECTOR ... ConstantFP:f32<-5.000000e-01>
fatal error: error in backend: Do not know how to soften this operator's operand!
```
Reciprocal refinement fails the same way above one refinement step. `DAGCombiner::BuildDivEstimate` builds a vector `1.0` constant. With one step, the final iteration uses the numerator instead, so the constant has no remaining uses and the case still compiles. With two or more steps, the constant remains live and exposes the illegal element type.
Square-root refinement builds constants whenever its refinement step count is positive. Reciprocal refinement needs a live constant only above one step. A fix therefore needs to preserve the currently working boundaries: zero-step square-root estimates, and zero-step or one-step reciprocal estimates.
Issue https://github.com/llvm/llvm-project/issues/217801 tracks the estimate node's own result type, a separate problem.
Contributor guide
Assessment
This issue has not been assessed yet.