cockroachdb / cockroachdb/apd

Precision clarification

Open
#121 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
805
Forks
48
PR merge metrics
No merged PRs in 30d

Description

I've been using the lib with a precision of 34, and I've noticed that when using `Text()`, some of my decimals terminate with `...9999999` or `..0000001`. I suspect it's got something to do with [this](https://github.com/cockroachdb/apd/blob/v2.0.2/format.go#L25).

Is there a way I can avoid having such errors, because my PoC app requires a certain sum total to be exactly zero, even at the precision of 34, which I can't achieve with such errors. I have to use `f` and can't use scientific notation. I'd like to know why this error occurs, and why it can't be fixed, because I've noticed the shopspring lib having a similar problem.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.