citusdata / citusdata/postgresql-hll
Setting regwidth >= 6 results in cardinality computed to NaN
Open
- Dominant language
- C
- Stars
- 1.2k
- Forks
- 115
- PR merge metrics
- No merged PRs in 30d
Description
Setting it to 5 works just fine, but setting it to 6 (Regardless log2m selected) always results in NaN.
Pulled the code locally to test the implementation against our implementation of HLL, and I noticed that issue. It's really hll.c with PG callback/integration bits removed.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.