boostorg / boostorg/math

Negative Binomial CDF and Quantile do not always roundtrip

Open
#935 25 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
374
Forks
264
Avg merge
1d 23h
Merged PRs (30d)
17

Description

Failure can be found here: https://github.com/scipy/scipy/pull/17432/checks?check_run_id=10961417302
Found on PR: https://github.com/scipy/scipy/pull/17432

Minimal reproducer:

```
import numpy as np
import numpy.testing as npt
from scipy import stats

dist = stats.nbinom(5, 0.5)
supp = 6
cdf_supp = dist.cdf(supp) # ideally 0.7255859375
cdf_supp0 = cdf_supp - 10*np.spacing(cdf_supp) # 0.7255859374999989
npt.assert_array_equal(dist.ppf(cdf_supp0), supp) # both should be 6
```

Note in their code says:

In very rare cases, the finite precision calculation of ppf(cdf(supp))
can produce an array in which an element is off by one. We nudge the
CDF values down by 10 ULPs help to avoid this.

At 15 ULPs it seems to be correct.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.