apache / apache/gluten

[VL][Spark-4.0] Add support for LeftSingle join for correlated scalar subqueries

Open
#11,372 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Description

Spark has introduced new join type `LeftSingle` which triggers changes in query execution in certain scenarios.

Single join is a left outer join that checks that there is at most 1 build row for every probe row.

Refer:
[[SPARK-49653](https://issues.apache.org/jira/browse/SPARK-49653)][SQL] Single join for correlated scalar subqueries
commit: https://github.com/apache/spark/commit/3c81f076ab9c72514cfc8372edd16e6da7c151d6

### Gluten version

None

Contributor guide

Open the contributing guide

Research direction

Start with the referenced SPARK-49653 issue and Spark commit 3c81f076ab9c72514cfc8372edd16e6da7c151d6, then trace Gluten’s Spark SQL execution path for correlated scalar subqueries. No file or test is named in the issue; completion means LeftSingle joins are supported in the affected execution scenarios without regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.