microsoft / microsoft/react-native-windows
JSValue conversion to double precision floating point gives wrong results
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 17.3k
- Forks
- 1.2k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 33
Description
In jsvalue.cpp we have:
static int64_t ToInt64(double value) noexcept {
return (std::numeric_limits<int64_t>::min() <= value && value <= std::numeric_limits<int64_t>::max())
? static_cast<int64_t>(value)
: 0;
}
However this seems wrong as the comparison between value and std::numeric_limits<int64_t>::max() is made by promoting max to double, which then loses precision:
https://godbolt.org/z/Ehxqef
int main(int argc, char**argv) {
std::cout << "int64 max = " << std::numeric_limits<int64_t>::max() << std::endl;
const double d = (double)(std::numeric_limits<int64_t>::max() - 511);
std::cout << "int64 max as double = " << std::setprecision(64) << d << std::endl;
return 0;
int64 max = 9223372036854775807
int64 max as double = 9223372036854775808
In fact, anything within +/- 511 of int64 max will be coerced onto a double with a value of 2^63. This is because the IEEE double mantissa is 52 bits, so anything above 2^52 will carry precision loss and may compare incorrectly.
Should we be limiting the ToInt64 function to only consider values within +-2^52 as valid in order to prevent loss of precision when converting to double? Or does it not matter?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in jsvalue.cpp at ToInt64 and reproduce the boundary behavior described in the issue, including values near int64_t's maximum and the double precision limit. Determine the intended valid range and verify that the chosen behavior handles the reported conversion cases without precision-related misclassification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100