Inconsistent (or buggy) Quantize behavior
- Dominant language
- Go
- Stars
- 805
- Forks
- 48
- PR merge metrics
- No merged PRs in 30d
Description
Looks like `Quantize(result, amount, 0)` func works differently for:
- when `amount` is of the form `N.0*` (where N is integer number and N != 0)
- when `amount` is of the form `0.0*`
Here is a simple test to reproduce this (with `v3.1.0`):
```go
// quantize is a simple wrapper around `apd` library.
func quantize(amount *apd.Decimal, decimalPlaces uint32, roundingRule apd.Rounder) (*apd.Decimal, error) {
ctx := apd.BaseContext.WithPrecision(infinitePrecision)
ctx.Rounding = roundingRule
var amt apd.Decimal
exp := -int32(decimalPlaces)
_, err := ctx.Quantize(&amt, amount, exp)
return &amt, err
}
func TestQuantizeRounding(t *testing.T) {
// 1.1 -> 2 (works as expected)
got, err = quantize(apd.New(11, -1), 0, apd.RoundUp)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want = apd.New(2, 0)
if want.CmpTotal(got) != 0 {
t.Fatalf("want: %+v, got: %+v", want, got)
}
// 1.01 -> 2 (works as expected)
got, err := quantize(apd.New(101, -2), 0, apd.RoundUp)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := apd.New(2, 0)
if want.CmpTotal(got) != 0 {
t.Fatalf("want: %+v, got: %+v", want, got)
}
// 0.1 -> 1 (works as expected)
got, err = quantize(apd.New(1, -1), 0, apd.RoundUp)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want = apd.New(1, 0)
if want.CmpTotal(got) != 0 {
t.Fatalf("want: %+v, got: %+v", want, got)
}
// 0.01 -> 1 (doesn't work as expected! instead it does 0.01 -> 0 for some reason)
got, err = quantize(apd.New(1, -2), 0, apd.RoundUp)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want = apd.New(1, 0)
if want.CmpTotal(got) != 0 {
t.Fatalf("want: %+v, got: %+v", want, got)
}
}
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the Context.Quantize entry point and reproduce the supplied TestQuantizeRounding cases against v3.1.0, focusing on values with small fractional exponents and RoundUp. Done means the 0.01 to 1 case behaves consistently with the other examples without regressing the existing cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100