apache / apache/datasketches-cpp
Weighted version of the KLL sketch?
- Dominant language
- C++
- Stars
- 273
- Forks
- 88
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 8
Description
Hello,
There's a consideration at [XGBoost](https://github.com/dmlc/xgboost/issues/5746) about potentially using the KLL sketch to represent feature value histograms.
One potential blocker is the need for a weighted version of the sketch, this would allow us to use data points that are weighted, and adjust their feature contributions accordingly (See Appendix A of [XGBoost paper](https://arxiv.org/abs/1603.02754)).
I remember discussing in the past the possibility of using data weights with KLL, is that still an option?
Contributor guide
Research direction
Start by reviewing the KLL sketch design and the linked XGBoost issue #5746, then read Appendix A of the linked XGBoost paper for the weighting requirements. Determine whether a weighted KLL sketch is feasible and define the expected behavior before implementation; the issue does not name source files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100