[APFloat] roundToIntegral / isInteger assert on Float8E3M4 and Float6E2M3FN
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
`IEEEFloat::roundToIntegral` adds and subtracts the magic constant
`2^(precision-1)`. For `Float8E3M4` (precision 5, largest 15.5) and
`Float6E2M3FN` (precision 4, largest 7.5) the constant itself overflows the
format, and `assert(fs == opOK)` fires on the `convertFromAPInt` of the
constant (APFloat.cpp:2452 at 5b5247c84485).
```c++
APFloat F(APFloat::Float8E3M4(), "1.0");
F.isInteger(); // Assertion `fs == opOK' failed.
```
Every finite value of these two formats is affected. With assertions off,
the constant saturates to the largest value and the result is wrong for
values with a fraction. `Float4E2M1FN` (precision 2, constant 2, largest 6)
is fine.
An early return for formats where the constant is not representable, or a
fallback that compares the value against the truncated integer, would fix
this.
Contributor guide
Research direction
Start in APFloat.cpp:2452 and reproduce the provided Float8E3M4 and Float6E2M3FN isInteger() example with assertions enabled. Trace roundToIntegral's magic-constant conversion, then verify that both formats avoid the assertion and return correct results for fractional finite values, while Float4E2M1FN remains unaffected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100