apache / apache/auron

Add join-time residual condition support for native semi and anti joins

Open
#2,196 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.8k
Forks
241
Avg merge
2d 14h
Merged PRs (30d)
22

Description

**Describe**
Semi and anti joins depend on whether a matching row satisfying the join condition exists. They cannot be implemented correctly by performing a native join and then filtering afterward. Auron currently rejects these joins when the Spark join condition is non-empty, which leaves `LeftSemi` and `LeftAnti` cases outside the native path.

**Describe the solution you'd like**
Add join-time residual condition support for native semi and anti joins.

This should build on the native join-time residual-condition mechanism introduced for outer joins and evaluate the condition during join matching so that existence semantics remain aligned with Spark.

Initial scope:
- `LeftSemi`
- `LeftAnti`

If `NullAwareAntiJoin` requires additional semantics beyond the initial implementation, it can remain on Spark fallback for the first version.

**Describe alternatives you've considered**
One alternative is to evaluate the residual predicate after a native semi/anti join, but that is not sufficient because semi/anti semantics depend on match existence during join evaluation.

Another alternative is to keep semi/anti joins with residual conditions on the Spark path until a fully generic join-condition framework is available, but that leaves an important class of Spark query rewrites unsupported.

**Additional context**
This issue should build on the join-time condition support introduced for outer joins, rather than introducing another separate mechanism.

This issue should not include:
- broadcast nested loop joins
- pure non-equi joins
- full null-aware anti join support unless it is small and clearly compatible with the initial design

Suggested validation:
- left semi join with `equi keys + residual predicate`
- left anti join with `equi keys + residual predicate`
- null-sensitive existence semantics
- clean fallback for unsupported residual predicates

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.