JuliaMath / JuliaMath/openlibm
Complete loss of precision for trig reduction in non-default rounding modes
Open
- Dominant language
- C
- Stars
- 632
- Forks
- 163
- Avg merge
- 4m
- Merged PRs (30d)
- 1
Description
The trig-reduction implementations assume that the rounding mode is round-to-nearest, and do not function correctly if it isn't. Example: sinf(-0x1.4665d2p+27). The mathematically precise result is approximately -0x1.927bcc77af475p-25, but in round-toward-zero or round-down, openlibm computes a result of -0x1.a2c7eep-17.
This behavior is especially bad for the "medium argument" path in rem_pio2f. When the argument is so large that you punt to rem_pio2, double-precision has enough extra bits to still deliver relatively good results.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.