citusdata / citusdata/citus

Perform Shard Pruning on Joins Between Compatible But Different Type Partition Columns

Open
#470 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

On joins between two tables where partition columns that have compatible but different types (for example: int-bigint, varchar-text), we don't do shard pruning, although we can.

I see that in JoinSequenceArray of multi_physical_planner, we perform a strict type comparison between two columns and decide that we cannot.

The behavior can be triggered with the following script:

```
CREATE TABLE itest1 (a int);
SELECT master_create_distributed_table('itest1', 'a', 'hash');
SELECT master_create_worker_shards('itest1', 64, 1);

CREATE TABLE itest2 (a bigint);
SELECT master_create_distributed_table('itest2', 'a', 'hash');
SELECT master_create_worker_shards('itest2', 64, 1);

SELECT * FROM itest1 JOIN itest2 ON itest1.a = itest2.a;
```

Contributor guide

Open the contributing guide

Research direction

Start in JoinSequenceArray of multi_physical_planner, where the issue reports strict type comparison for join columns. Reproduce the behavior with the provided itest1/int and itest2/bigint SQL script, then trace the compatible-type check. Done means the join performs shard pruning when partition columns have compatible but different types.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql, sql
Domain
databases, distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.