apache / apache/datasketches-cpp

Weighted version of the KLL sketch?

Open
#157 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
273
Forks
88
Avg merge
2d 2h
Merged PRs (30d)
8

Description

Hello,

There's a consideration at [XGBoost](https://github.com/dmlc/xgboost/issues/5746) about potentially using the KLL sketch to represent feature value histograms.

One potential blocker is the need for a weighted version of the sketch, this would allow us to use data points that are weighted, and adjust their feature contributions accordingly (See Appendix A of [XGBoost paper](https://arxiv.org/abs/1603.02754)).

I remember discussing in the past the possibility of using data weights with KLL, is that still an option?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the KLL sketch design and the linked XGBoost issue #5746, then read Appendix A of the linked XGBoost paper for the weighting requirements. Determine whether a weighted KLL sketch is feasible and define the expected behavior before implementation; the issue does not name source files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.