4paradigm / 4paradigm/OpenMLDB

Add optimization passes for native LastJoin

Open
#413 1 comment 0 reactions 1 assignee Claimed by @tobegit3hub View on GitHub
enhancement
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

Native LastJoin is implemented in OpenMLDB Spark distribution and make great performance improved. We should add more optimization passes like LeftOuterJoin to leverage the improvement of Spark.

Contributor guide

Open the contributing guide

Research direction

Look at the existing native LastJoin implementation in the OpenMLDB Spark distribution to understand its optimization passes. Investigate how LeftOuterJoin and similar joins are currently handled. Determine what changes are needed to extend the optimization passes, likely in the Spark-related codebase within the project. Testing will involve verifying performance improvements for the new optimization passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark
Domain
backend, databases, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.