NVIDIA / NVIDIA/cccl

Harmonize methods used for floating-point approximate equality testing across CCCL test suite

Open
#7,662 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

Presently, CCCL uses different tests for approximate equality of compute result against a reference value.

CCCL uses the following to test proximity to reference result

$$
\vert a - b \vert \le a_\{\mathrm{tol}} + r_{\mathrm{tol}} \cdot \mathrm{scale}(a, b)
$$

where $a_{\mathrm{tol}}$ and $r_{\mathrm{tol}}$ values used across different constituents of CCCL may differ. Moreover, $\mathrm{scale}(a, b)$ is sometimes defined as $\vert a \vert + \vert b \vert$, and sometimes as $\vert b \vert$.

The latter choice makes the testing function $\mathrm{isclose}(a, b)$ non-symmetric and should be avoided when testing value computed on GPU against value computed on CPU (as opposed to checking for reference value known a-priori).

[PEP-0485](https://peps.python.org/pep-0485/) proposes to use the following test, which is explicitly symmetric and has intuitive meaning of tolerance parameters:

$$
\mathrm{isclose}(a, b) := (\vert a - b \vert \le \max( a_{\mathrm{tol}}, r_{\mathrm{tol}} \cdot \max( \vert a \vert, \vert b \vert )) )
$$

Furthermore, choice of tolerance parameters should depend on `std::numeric_limits::epsilon()` and be based on Wilkinson-type error estimates derived for the pertinent algorithm.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.