cockroachdb / cockroachdb/apd

Ln func discrepancies across arch

Open
#141 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
805
Forks
48
PR merge metrics
No merged PRs in 30d

Description

## Description

When using `Context#Ln()` func we can observe different results depending on the system's arch, for example with the sample code below:

```go
var decCtx = &apd.Context{
Precision: 34,
MaxExponent: apd.MaxExponent,
MinExponent: apd.MinExponent,
Traps: apd.DefaultTraps,
Rounding: apd.RoundDown,
}
i, _, _ := apd.NewFromString("1.6285091944505809264504560045920167")
var r apd.Decimal
decCtx.Ln(&r, i)
fmt.Println(r.String())
```

The output on arm64 is `0.4876649916811116824516548471782886` and `0.4876649916811116824516548471782887` on amd64.

By investigating it seems the root cause is linked to the usage of `math.Log()` [here](https://github.com/cockroachdb/apd/blob/9cfbeb8698620942481da9e7fb4d46b5ac4df03d/context.go#L755) which is dependent on the system arch.

Is this difference expected? Is there a mean to mitigate this and have consistent results across arch?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.