[APFloat] toString with an explicit FormatPrecision truncates the last digit for some values
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
`AdjustToPrecision(APInt &significand, int &exp, unsigned FormatPrecision)`
in `llvm/lib/Support/APFloat.cpp` divides the significand by a power of ten
(integer division) to reduce the digits before the decimal conversion. It
removes `floor((bits - bitsRequired) * 59 / 196)` decimal digits, which
leaves `FormatPrecision` or `FormatPrecision + 1` digits. When exactly
`FormatPrecision` digits remain, the digit-level rounding in the second
`AdjustToPrecision` has no guard digit, and the result is truncated.
```
APFloat F(8999999488.0f); // 0x50061C46
SmallString<16> S;
F.toString(S, /*FormatPrecision=*/1, /*FormatMaxPadding=*/0, false);
// S == "8.0e+09"; the correctly rounded result is "9.0e+09".
F.toString(S, 6, 0, false);
// S == "8.99999e+09"; the correctly rounded result is "9.00000e+09".
```
Over 20000 random doubles and `FormatPrecision` 1..17, 2.5% of the
strings differ from the correctly rounded (half-up) result. The natural
precision (FormatPrecision = 0) still round-trips because it has spare
digits, but `toStringShortest` (#218471) returns one digit more than the
optimum for about 0.15% of doubles, and the MLIR/IR printers fall back to
hex for constants whose 6-digit form would round-trip if rounded.
A fix is to keep one more decimal digit (decrement `tensRemovable` by one),
so the digit-level rounding always sees a guard digit. This changes
`toString` output at explicit precisions wherever it truncated, so tests
in MLIR, llubi, and clang diagnostics need to be checked.
Contributor guide
Research direction
Start in llvm/lib/Support/APFloat.cpp at AdjustToPrecision and trace the two precision-adjustment steps, then reproduce the provided APFloat examples. Verify that explicit-precision output is correctly rounded with a guard digit, and check the mentioned MLIR, llvm, and clang diagnostic tests for expected output changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100