KhronosGroup / KhronosGroup/GLSL
significand range for frexp() incorrect due to sign-handling
- Dominant language
- JavaScript
- Stars
- 458
- Forks
- 114
- Avg merge
- 4m
- Merged PRs (30d)
- 1
Description
The specification says the range of the significand (aka mantissa) returned by ``frexp()`` is **[0.5,1.0]**. Handling of the sign of the floating point number is not mentioned except for the special case of minus zero.
In C/C++ ``frexp()`` preserves the sign in the significand, so the range is documented as **[-1.0,-0.5)** and **(0.5,1.0]**. NVIDIA ``frexp()`` appears to preserve the sign in the significand. Given minus zero preservation, I believe sign-preservation was the intent and the specification is in error.
Related, the C/C++ ranges are documented as exclusive/inclusive to guarantee you won't get 0.5 as a significand. The inclusive-inclusive range in the GLSL specification deviates from this, presumably intentionally to allow flexibility in the implementation. (For example, it appears that the NVIDIA GPU on this desktop returns **0.5** for ``frexp(1.0,exp)``.) In my opinion it would be nice to be explicit about this deviation from C/C++ instead of expecting people to infer an unusual behavior from a single unemphasized character being '**[**' instead of '**(**'.
Also related, ARB_gpu_shader5 documents the range as **[0.5,1.0)**, with the inclusive-exclusive reversed from C/C++. Again presumably intentionally, but I imagine it should be made consistent with the GLSL specification if it gets the sign fix.
To summarize, the following are the significand ranges found for ``frexp()`` in various places:
**[0.5,1.0]** - GLSL 4.60 specification (and earlier)
**[0.5,1.0)** - ARB_gpu_shader5 version 16 (2012)
**(0.5,1.0]** & **[-1.0,-0.5)** - C/C++
**[0.5,1.0]** & **[-1.0,-0.5]** - proposed corrected GLSL range
(Note that I have reversed the order of the pairs of ranges above from their natural ordering to make visual comparison simpler.)
The handling of sign MUST be corrected, and the inconsistency with C/C++ SHOULD be documented explicitly.
Contributor guide
Assessment
This issue has not been assessed yet.