Replace Datasketch HLL with newer Datasketch CPC sketch
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Motivation
the datasketches library has a new Unique Counting Sketch called CPC sketch that has better accuracy per size than HLL.
https://github.com/DataSketches/sketches-core/releases
### Proposed changes
replace Datasketch HLL sketch with CPC or offer it alongside as a higher accuracy sketch
### Rationale
Better accuracy
### Operational impact
The docs of datasketches don't describe the whether CPC algorithm is more CPU intensive or not. This will determine whether we want to completely deprecate DataSketches HLL sketch and replace it with CPC or keep CPC as an additional option.
Contributor guide
Research direction
Start by reviewing the linked DataSketches releases and the existing Druid integration for the HLL sketch; the issue does not name files or tests. Compare CPC and HLL accuracy and CPU impact, then determine whether CPC should replace HLL or be offered alongside it. Done means the chosen integration and its operational trade-offs are documented and covered by appropriate tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100