Perform Shard Pruning on Joins Between Compatible But Different Type Partition Columns
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
On joins between two tables where partition columns that have compatible but different types (for example: int-bigint, varchar-text), we don't do shard pruning, although we can.
I see that in JoinSequenceArray of multi_physical_planner, we perform a strict type comparison between two columns and decide that we cannot.
The behavior can be triggered with the following script:
```
CREATE TABLE itest1 (a int);
SELECT master_create_distributed_table('itest1', 'a', 'hash');
SELECT master_create_worker_shards('itest1', 64, 1);
CREATE TABLE itest2 (a bigint);
SELECT master_create_distributed_table('itest2', 'a', 'hash');
SELECT master_create_worker_shards('itest2', 64, 1);
SELECT * FROM itest1 JOIN itest2 ON itest1.a = itest2.a;
```
Contributor guide
Research direction
Start in JoinSequenceArray of multi_physical_planner, where the issue reports strict type comparison for join columns. Reproduce the behavior with the provided itest1/int and itest2/bigint SQL script, then trace the compatible-type check. Done means the join performs shard pruning when partition columns have compatible but different types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, postgresql, sql
- Domain
- databases, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100