JuliaMath / JuliaMath/FixedPointDecimals.jl

Subtraction and comparison against floats is not robust against float underflow

Open
#117 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
37
Forks
23
PR merge metrics
No merged PRs in 30d

Description

```julia
julia> using FixedPointDecimals

julia> fd = reinterpret(FixedDecimal{Int,0}, (2 << 53) + 1)
FixedDecimal{Int64,0}(18014398509481985)

julia> f = Float64(2 << 53)
1.8014398509481984e16

julia> isless(f, fd) # this is basically asking 18014398509481984 < 18014398509481985 -> true
false

julia> fd - f # this is basically 18014398509481985 - 18014398509481984 -> 1
0.0
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.