4paradigm / 4paradigm/OpenMLDB

Support null equal in last join for openmldb-batch

Open
#1,193 0 comments 0 reactions 1 assignee Claimed by @tobegit3hub View on GitHub
enhancement
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

We may support `===` and `<=>` for last join.

```
--- a/java/openmldb-batch/src/main/scala/com/_4paradigm/openmldb/batch/nodes/JoinPlan.scala
+++ b/java/openmldb-batch/src/main/scala/com/_4paradigm/openmldb/batch/nodes/JoinPlan.scala
@@ -91,10 +91,18 @@ object JoinPlan {
for (i <- 0 until keyNum) {
val leftColumn = SparkColumnUtil.resolveExprNodeToColumn(leftKeys.GetChild(i), node.GetProducer(0), leftDf)
val rightColumn = SparkColumnUtil.resolveExprNodeToColumn(rightKeys.GetChild(i), node.GetProducer(1), rightDf)
- joinConditions += (leftColumn === rightColumn)
+ joinConditions += leftColumn.eqNullSafe(rightColumn)
}
}
```

Contributor guide

Open the contributing guide

Research direction

The change is in JoinPlan.scala, specifically the join condition logic. Start by understanding the existing last join implementation and how null equality is currently handled. Look at the SparkColumnUtil and the eqNullSafe method. Run tests related to join operations to verify the new behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
databases, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.