How to use Quantize with limited precision context
- Lingua principale
- Go
- Stelle
- 805
- Fork
- 48
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
I'm building a financial application that deals with multiple currencies and multiple Contexts with settings appropriate to each currency. The DB storage supports the widest possible range of values to permit to store any currency, DB storage uses Decimal(78,18), i.e. `MaxExponent=60` and `MinExponent=-18` in `apd` terms.
I'm trying to quantize the value coming from DB with highest possible precision into the precision appropriate for given currency. The value is not large, but has many trailing decimal zeroes that I want to remove to bring it to correct precision.
And for whatever reason `Quantize()` does not produce the result I expect, but throws an `Overflow` condition instead. Below is the sample code that reproduces it:
```
package main
import (
"fmt"
"github.com/cockroachdb/apd/v2"
)
func main() {
dStr := "6410.000000000000000000" // small value with many trailing zeroes as seen in DB
wideCon := &apd.Context{ // widest supported context
MaxExponent: 78,
MinExponent: -18,
Traps: apd.DefaultTraps,
}
d, _, _ := wideCon.SetString(&apd.Decimal{}, dStr)
limitCon := &apd.Context{ // limited context suitable for given currency
Precision: 17,
MaxExponent: 9,
MinExponent: -8,
Traps: apd.DefaultTraps,
}
scratch := &apd.Decimal{}
con, err := limitCon.Quantize(scratch, d, limitCon.MinExponent) // attempt to quantize to correct precision, expect to work fine
fmt.Println(con, err, scratch) // fails with `overflow` condition
}
```
In this piece of code the `wideCon` is the widest supported context in the system, i.e. the DB default context, while `limitCon` is the target currency context. If I understand correctly the context settings - the `limitCon` effectively forces the values in this context to be within `999999999.99999999` range, which is exactly what I need.
The sample value is way below the `MaxExponent` limit, and I expect `limitCon.Quantize()` to produce same value with decimal places adjusted to fit, but instead it returns a `NaN` value and an `Overflow` condition.
The error goes away if I increase the `MaxExponent`, it works at `MaxExponent=12` for this value, but workarounding it this way means I will have to forfeit the ceiling check limit which I'd like to not do.
What is a better way to achieve what I need - round an arbitrary long precision value to a constrained precision settings force-rounded per currency and with correct ceiling constraint?
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.