JuliaMath / JuliaMath/openlibm

Complete loss of precision for trig reduction in non-default rounding modes

Open
#30 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
632
Forks
163
Avg merge
4m
Merged PRs (30d)
1

Description

The trig-reduction implementations assume that the rounding mode is round-to-nearest, and do not function correctly if it isn't. Example: sinf(-0x1.4665d2p+27). The mathematically precise result is approximately -0x1.927bcc77af475p-25, but in round-toward-zero or round-down, openlibm computes a result of -0x1.a2c7eep-17.

This behavior is especially bad for the "medium argument" path in rem_pio2f. When the argument is so large that you punt to rem_pio2, double-precision has enough extra bits to still deliver relatively good results.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.