llvm / llvm/llvm-project

[APFloat] PPCDoubleDouble overflow under rmTowardZero: convertFromString asserts, add/multiply return +Inf

Open
#223,478 0 comments 0 reactions 0 assignees View on GitHub
llvm:support
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

With `PPCDoubleDouble` and `rmTowardZero`, two overflow paths misbehave (both reproduced on main, independent of #220020):

1. `convertFromString("1e400", rmTowardZero)` hits an assertion:

```
APFloat.cpp:3489: llvm::APInt llvm::detail::IEEEFloat::convertPPCDoubleDoubleLegacyAPFloatToAPInt() const: Assertion `fs == opOK || fs == opInexact' failed.
```

The string is converted in the legacy 106-bit semantics, which rounds toward zero to its largest value. `convertPPCDoubleDoubleLegacyAPFloatToAPInt` then converts that to `IEEEdouble` with `rmNearestTiesToEven`, which rounds up past `DBL_MAX` and overflows.

2. `getLargest() + getLargest()` and `getLargest() * getLargest()` under `rmTowardZero` return `+Inf` with only `opInexact` set. Rounding toward zero should never produce an infinity from finite operands; `IEEEdouble` gives the largest finite value here.

Reproducer:

```cpp
#include "llvm/ADT/APFloat.h"
#include "llvm/ADT/SmallString.h"
#include "llvm/Support/Error.h"
#include
using namespace llvm;

static void show(const char *Op, APFloat::opStatus St, const APFloat &V) {
SmallString<48> S;
V.toString(S);
printf("%-16s %s overflow=%d inexact=%d\n", Op, S.c_str(),
!!(St & APFloat::opOverflow), !!(St & APFloat::opInexact));
}

int main() {
setvbuf(stdout, nullptr, _IONBF, 0); // keep the output before the assert
const fltSemantics &Sem = APFloat::PPCDoubleDouble();
auto RM = APFloat::rmTowardZero;

APFloat A = APFloat::getLargest(Sem);
show("max + max", A.add(APFloat::getLargest(Sem), RM), A);

APFloat M = APFloat::getLargest(Sem);
show("max * max", M.multiply(APFloat::getLargest(Sem), RM), M);

APFloat F(Sem);
auto St = F.convertFromString("1e400", RM); // asserts
if (!St)
consumeError(St.takeError());
else
show("fromString 1e400", *St, F);
}
```

Current output (assertions enabled):

```
max + max +Inf overflow=0 inexact=1
max * max +Inf overflow=0 inexact=1

```

Expected: the largest finite `PPCDoubleDouble` value in all three cases, and no assertion. (Whether `opOverflow` is also set depends on #220020, which makes `IEEEFloat` signal it on this path.)

For comparison, `convertFromAPInt(2^1100, false, rmTowardZero)` on `PPCDoubleDouble` does return the largest finite value (`1.79769313486231580793728971405301E+308`), through `DoubleAPFloat::handleOverflow`.

Contributor guide

Open the contributing guide

Research direction

Start in APFloat.cpp at convertPPCDoubleDoubleLegacyAPFloatToAPInt and the PPCDoubleDouble overflow paths used by add, multiply, and convertFromString. Compare these with convertFromAPInt and DoubleAPFloat::handleOverflow, then reproduce the three cases from the issue. Done means finite largest-value results for all three rmTowardZero cases without the assertion, with status behavior covered as appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.