llvm / llvm/llvm-project

[NSAN] wrong value in "hex" for the float128

Open
#164,247 1 comment 0 reactions 2 assignees Claimed by @legrosbuffle View on GitHub
compiler-rt:nsan
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

example
WARNING: NumericalStabilitySanitizer: inconsistent shadow results while checking call argument #0
double precision (native): dec: -0.00013593408999037200 hex: -0x1.1d130f11c72c70000000p-13
__float128 precision (shadow): dec: -0.00013593438856027846 hex:-0x8.e899c0d62e8031700000p-16
shadow truncated to double : dec: -0.00013593438856027845 hex: -0x1.1d13381ac5d000000000p-13
Relative error: 0.00021964265968929818% (2^33 epsilons)

it seems to me that "hex" value in the second line is wrong (it should start with the very same digits of the other two as the decimal values do)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.