apache / apache/pinot

[Query Optimizer Improvement] IN statement for timestamp applied as pruning mechanism properly

Open
#6,626 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Right now if we write something like `SELECT * FROM fact WHERE primary_key_id in ('id1', 'id2', 'id3', .....'id10' ) and created_at in (ts1, ts2, ts3, ts4, ... ts10) LIMIT 2000` it is actually running slower than not passing in `created_at in (xxx....)` filter.
But if we convert the `created_at in (xxx) ` into a range filter, it will run faster. Converted query is like below

`SELECT * FROM fact WHERE primary_key_id in ('id1', 'id2', 'id3', .....'id10' ) and created_at <= 1614426347 and created_at >= 1614426293 LIMIT 2000`

Contributor guide

Open the contributing guide

Research direction

No source files, tests, or entry points are named in the issue. Start by reproducing the timestamp IN query and its range-filter equivalent, then trace query-optimizer pruning for timestamp predicates; done means the IN form prunes effectively and does not regress relative to the range form.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.