apache / apache/datasketches-bigquery

Consider adding theta_sketch_agg_int64_lgk()

Open
#144 13 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
21
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Currently there are 2 methods to create theta sketches for int64 :

- theta_sketch_agg_int64(value INT64)
- theta_sketch_agg_int64_lgk_seed_p(value INT64, params STRUCT NOT AGGREGATE)

If I need to increase precision of the sketch, then I am forced to pass a seed and p value.
If a sketch is created with a seed, then end users now cannot use theta_sketch_agg_union(sketch BYTES) anymore and are forced to remember the seed value and use theta_sketch_agg_union_lgk_seed(sketch BYTES, params STRUCT NOT AGGREGATE)

Using theta_sketch_agg_union() on a sketch initialized with a seed gives me following error:
```
seed hash mismatch: expected 37836, actual 54156 at bqutil.datasketches.theta_sketch_agg_union_lgk_seed(BYTES, STRUCT) line 55, columns 6-7; reason: invalidQuery, location: query, message: Error: seed hash mismatch: expected 37836, actual 54156 at bqutil.datasketches.theta_sketch_agg_union_lgk_seed(BYTES, STRUCT) line 55, columns 6-7
```

Can we consider adding a function`theta_sketch_agg_int64_lgk((value INT64, lg_k INT64)` ?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the existing theta_sketch_agg_int64(value INT64) and theta_sketch_agg_int64_lgk_seed_p(value INT64, params STRUCT NOT AGGREGATE) entry points. Review how theta_sketch_agg_union() and theta_sketch_agg_union_lgk_seed() handle sketch parameters and the reported seed hash mismatch. Done means the requested lg_k-only function supports increased precision without requiring callers to provide seed and p values.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.