microsoft / microsoft/react-native-windows

JSValue conversion to double precision floating point gives wrong results

Open
#5,622 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Code Cleanup Workstream: Component Parity
Dominant language
C++
Stars
17.3k
Forks
1.2k
Avg merge
1d 13h
Merged PRs (30d)
33

Description

In jsvalue.cpp we have:

  static int64_t ToInt64(double value) noexcept {
    return (std::numeric_limits<int64_t>::min() <= value && value <= std::numeric_limits<int64_t>::max())
        ? static_cast<int64_t>(value)
        : 0;
  }

However this seems wrong as the comparison between value and std::numeric_limits<int64_t>::max() is made by promoting max to double, which then loses precision:
https://godbolt.org/z/Ehxqef

int main(int argc, char**argv) {
std::cout << "int64 max = " << std::numeric_limits<int64_t>::max() << std::endl;
const double d = (double)(std::numeric_limits<int64_t>::max() - 511);
std::cout << "int64 max as double = " << std::setprecision(64) << d << std::endl;
 
return 0;
int64 max = 9223372036854775807
int64 max as double = 9223372036854775808

In fact, anything within +/- 511 of int64 max will be coerced onto a double with a value of 2^63. This is because the IEEE double mantissa is 52 bits, so anything above 2^52 will carry precision loss and may compare incorrectly.

Should we be limiting the ToInt64 function to only consider values within +-2^52 as valid in order to prevent loss of precision when converting to double? Or does it not matter?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in jsvalue.cpp at ToInt64 and reproduce the boundary behavior described in the issue, including values near int64_t's maximum and the double precision limit. Determine the intended valid range and verify that the chosen behavior handles the reported conversion cases without precision-related misclassification.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.