google / google/double-conversion

Calling DoubleToStringConverter::ToShortest with the value 1.80113 generates the string 1.8011300000000001

Open
#84 3 comments 0 reactions 1 assignee Claimed by @floitsch View on GitHub
Dominant language
C++
Stars
1.2k
Forks
308
Avg merge
2d 9h
Merged PRs (30d)
12

Description

The produced output 1.8011300000000001 is at least not what I would expect from ToShortest.

This code path doesn't generate a fallback to bignum calculations. I can't fully deduce from the comments whether this is intentionally.

But the net outcome is that this makes ToShortest a no go for our usage.

We have implemented an overlay of ToShortest that takes an additional max_digits input that we can use to solve the problem, but I'm wondering whether there is a better / more general solution.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.