JuliaMath / JuliaMath/FixedPointNumbers.jl

Wrong printing of `Fixed` numbers

Open
#307 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
95
Forks
38
Avg merge
13m
Merged PRs (30d)
2

Description

Either I have totally misunderstood the documentation or printing of `Fixed` numbers is wrong.

```
julia> Fixed{Int8,2}(0.25)
0.2Q5f2

julia> Fixed{Int8,3}(0.125)
0.12Q4f3
```

Looking into `FixedPointNumbers.jl/show` leads to
```
digits=ceil(Int, f * log10_2)
```
which is wrong, as for `2^-f` the number of decimal fraction digits scales linearly and not logarithmic with `f`'s number of binary fraction digits, as you can see here
```
julia> 2.0.^(-1:-1:-10)
10-element Vector{Float64}:
0.5
0.25
0.125
0.0625
0.03125
0.015625
0.0078125
0.00390625
0.001953125
0.0009765625
```

Consequently this can be fixed by using `digits=f`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.