llvm / llvm/llvm-project

Provide proof that `tan(x)` does not overflow to infinity for `fp80`, `fp128`, and `ppcf128`.

Open
#219,714 1 comment 0 reactions 0 assignees View on GitHub
floating-point llvm:support
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Currently, no proof is provided that `tan(x)` never overflows to infinity in `KnownFPClass::tan`.
https://github.com/llvm/llvm-project/blob/25525299adf55ea46d8590ef4385ea66f2679822/llvm/lib/Support/KnownFPClass.cpp#L664-L669

You can brute force prove that a correctly rounded `tan(float)` will never overflow to infinity, and it seems like `tan(double)` will never overflow either https://stackoverflow.com/questions/67482420/proof-of-impossibility-or-counterexample-for-tanx-infinity-for-floating-p

However, it seems like there is no proof that `tan(x)` won't overflow for `f80`, `f128`, or `ppcf128`. Also, since `f80` represents a subset of the values from `f128`, a proof that `tan(f128)` never overflows automatically proves that `tan(f80)` will also never overflow. If I were to guess, `ppcf128` is probably the most likely to overflow since it has the smallest exponent.

I suspect the existing deduction is correct, but brute-force testing cannot rule out an exceptionally close representable approximation to a pole for `tan(x)`.

***

Using this AI assisted script, I brute force checked all poles up to 100'000'000.5 * pi for `ppcf128` at 160 bits of precision. The script would compute `tan(x)` where `x` is `(pole_number + 0.5) * pi` rounded to the nearest `ppcf128` value.
https://pastebin.com/KeaqrwTq

This result was the closest to overflowing that I could find (required 112 bits of precision to detect, and I verified the result at 2112 bits of precision)
```
x = 5564510.5 * pi
x = 0x1.0abed14ec066p+24 + -0x1.83ec193dafea8p-38
tan(x) = 0xb.44d4b3884399665e566777e455a18e8e2ecf841bp+108
```

Contributor guide

Open the contributing guide

Research direction

The target is llvm/lib/Support/KnownFPClass.cpp, especially KnownFPClass::tan at lines 664-669. Start by tracing the existing no-overflow deduction and comparing the representable ranges of fp80, fp128, and ppcf128, using the issue's pole-search result as context. Done means the deduction is backed by a valid proof for all three formats, or is no longer asserted where it remains unproven.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.