[multistage] physical optimizer lookup join support
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
While testing the physical optimizer, we get this error when trying to invoke a lookup join:
Error log:
```
"message": "Received 1 error from stage N on Server_1234: Right input must be leaf operator",
```
Query:
```
WITH "source_1" AS (
SELECT
'v1' AS "column_1",
1234567890 AS "column_2"
FROM "table_3"
LIMIT 1
)
SELECT
/*+ "joinOptions"("join_strategy" = 'lookup') */
a."column_1",
a."column_2",
b."column_3"
FROM "source_1" a
INNER JOIN "table_2" b
ON b."column_1" = a."column_1"
AND b."column_2" = a."column_2"
```
Claude's analysis:
> The V2 physical optimizer (usePhysicalOptimizer=true) does not correctly handle lookup joins. When a query uses /*+ joinOptions(join_strategy='lookup') */, V2 inserts a BROADCAST exchange on the right side (dim table), which splits the dim table scan into a separate fragment from the join. At runtime, LookupJoinOperator requires rightInput instanceof LeafOperator but receives a MailboxReceiveOperator instead, throwing "Right input must be leaf operator"
cc @ankitsultana @suvodeep-pyne
Contributor guide
Assessment
This issue has not been assessed yet.