apache / apache/cloudberry

Should we support other scan types than SeqScan for runtime filter?

Open
#1,234 6 comments 0 reactions 0 assignees View on GitHub
type: Enhancement type: Performance
Dominant language
C
Stars
1.4k
Forks
247
Avg merge
4d 3h
Merged PRs (30d)
39

Description

Currently, runtime filter only support Seqscan and Dynamic Seqscan, which has a positive performance gain for HashJoin, because it reduces probe HashTable cost when doing join by bloom filter.

For example,
```
postgres=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF)
SELECT t1.c3 FROM t1, t2 WHERE t1.c2 = t2.c2;
QUERY PLAN
-------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (actual rows=0 loops=1)
-> Hash Join (actual rows=0 loops=1)
Hash Cond: (t1.c2 = t2.c2)
Extra Text: (seg2) Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
-> Seq Scan on t1 (actual rows=1 loops=1)
Rows Removed by Pushdown Runtime Filter: 127
-> Hash (actual rows=32 loops=1)
Buckets: 524288 Batches: 1 Memory Usage: 4098kB
-> Seq Scan on t2 (actual rows=32 loops=1)
Optimizer: Postgres query optimizer
(10 rows)
```

For HashJoin, the scan could be FunctionScan, SubqueryScan, TidScan and etc, so should we support these Scan types, shall we have performance gain?

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by tracing runtime-filter handling for SeqScan and Dynamic Seqscan around HashJoin, then compare FunctionScan, SubqueryScan, and TidScan; done means defining the supported scan types and providing evidence of any performance gain.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.