Cross-schema/cross-catalog qualified column doesn't do ambiguity check
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
If joining two identical tables from different schemas, and selecting a column using a table qualifier as part of the identifier, it should do ambiguity check and fail if referring to an ambiguous column.
### To Reproduce
Via datafusion-cli:
```sql
DataFusion CLI v22.0.0
❯ create schema s1;
0 rows in set. Query took 0.040 seconds.
❯ create schema s2;
0 rows in set. Query took 0.001 seconds.
❯ create table s1.t as select 1 as a, 2 as b;
0 rows in set. Query took 0.033 seconds.
❯ create table s2.t as select 1 as a, 2 as b;
0 rows in set. Query took 0.003 seconds.
❯ select t.b from s1.t join s2.t using (a);
+---+
| b |
+---+
| 2 |
+---+
1 row in set. Query took 0.039 seconds.
❯ explain select t.b from s1.t join s2.t using (a);
+---------------+----------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+---------------+----------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: s1.t.b |
| | Inner Join: Using s1.t.a = s2.t.a |
| | TableScan: s1.t projection=[a, b] |
| | TableScan: s2.t projection=[a] |
| physical_plan | ProjectionExec: expr=[b@1 as b] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | HashJoinExec: mode=Partitioned, join_type=Inner, on=[(Column { name: "a", index: 0 }, Column { name: "a", index: 0 })] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | RepartitionExec: partitioning=Hash([Column { name: "a", index: 0 }], 12), input_partitions=1 |
| | MemoryExec: partitions=1, partition_sizes=[1] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | RepartitionExec: partitioning=Hash([Column { name: "a", index: 0 }], 12), input_partitions=1 |
| | MemoryExec: partitions=1, partition_sizes=[1] |
| | |
+---------------+----------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.031 seconds.
❯
```
Can see there is identical table `t` in both schemas `s1` and `s2`, and selecting `t.b` column (see it's qualified with table) should do ambiguity check as could be in either table.
To note, if column is not qualified at all and left as `b` then ambiguity check will occur and return error.
Similarly if schema & table are identical but in separate catalogs, issue also occurs:
```sql
DataFusion CLI v22.0.0
❯ create database d1;
0 rows in set. Query took 0.001 seconds.
❯ create database d2;
0 rows in set. Query took 0.001 seconds.
❯ create schema d1.s;
0 rows in set. Query took 0.001 seconds.
❯ create schema d2.s;
0 rows in set. Query took 0.001 seconds.
❯ create table d1.s.t as select 1 as a, 2 as b;
0 rows in set. Query took 0.002 seconds.
❯ create table d2.s.t as select 1 as a, 2 as b;
0 rows in set. Query took 0.002 seconds.
❯ select t.b from d1.s.t join d2.s.t using (a);
+---+
| b |
+---+
| 2 |
+---+
1 row in set. Query took 0.006 seconds.
❯ explain select t.b from d1.s.t join d2.s.t using (a);
+---------------+----------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+---------------+----------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: d1.s.t.b |
| | Inner Join: Using d1.s.t.a = d2.s.t.a |
| | TableScan: d1.s.t projection=[a, b] |
| | TableScan: d2.s.t projection=[a] |
| physical_plan | ProjectionExec: expr=[b@1 as b] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | HashJoinExec: mode=Partitioned, join_type=Inner, on=[(Column { name: "a", index: 0 }, Column { name: "a", index: 0 })] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | RepartitionExec: partitioning=Hash([Column { name: "a", index: 0 }], 12), input_partitions=1 |
| | MemoryExec: partitions=1, partition_sizes=[1] |
| | CoalesceBatchesExec: target_batch_size=8192 |
| | RepartitionExec: partitioning=Hash([Column { name: "a", index: 0 }], 12), input_partitions=1 |
| | MemoryExec: partitions=1, partition_sizes=[1] |
| | |
+---------------+----------------------------------------------------------------------------------------------------------------------------+
2 rows in set. Query took 0.007 seconds.
❯
```
### Expected behavior
Should return error about ambiguous column
### Additional context
Ambiguity check was fixed in https://github.com/apache/arrow-datafusion/pull/5509 but seems this only accounted for unqualified columns, not qualified ones as well.
Contributor guide
Research direction
Reproduce the qualified-column and unqualified-column cases with datafusion-cli, then inspect the ambiguity check introduced by PR 5509. Trace how qualified identifiers are resolved for joins across schemas or catalogs, and add coverage demonstrating that t.b returns an ambiguity error; done means both reported examples fail as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100