llnl / llnl/zfp

Tiny but above subnormal numbers not handled correctly

Open
#209 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
885
Forks
181
PR merge metrics
No merged PRs in 30d

Description

As mentioned in #119, blocks with all subnormal but nonzero numbers are not encoded correctly because of reciprocal overflow, which can be avoided using the `ZFP_WITH_DAZ` compile-time macro. However, there are even larger, normal numbers that cause problems for zfp due to how they are currently converted to integers here:
https://github.com/LLNL/zfp/blob/c1845815ef1068b578b32823fb22310bd0b62b56/src/template/encodef.c#L42-L59
When the largest (in magnitude) normal value in a block is strictly smaller than 2-98 ≈ 3.2e-30 for floats or 2-962 ≈ 2.6e-290 for doubles, overflow in the computation of *s* occurs (even if `ZFP_WITH_DAZ` is enabled). While very small, both of these numbers are well above the smallest normal numbers `FLT_MIN` = 2-126 and `DBL_MIN` = 2-1022, respectively.

While less performant, a potential solution is to make use of division by 1 / *s* instead of multiplication by *s* for numbers in this range by computing 1 / *s* via `ldexp` directly, which is guaranteed not to underflow. Note that *s* (and 1 / *s*) is always an integer power of two, so the same result is obtained whether division or multiplication is used (when no overflow occurs). This solution is more general than `ZFP_WITH_DAZ`, as it also correctly handles all-subnormals.

Contributor guide

Open the contributing guide

Research direction

Start by reading src/template/encodef.c at lines 42-59 and the issue's discussion of ZFP_WITH_DAZ and powers-of-two scaling. Investigate the conversion and scaling path for very small normal and all-subnormal values, then verify that those ranges no longer overflow while ordinary values retain the same result.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.