apache / apache/hudi

Avoid using Projections in generic Merge Into DMLs

Open
#15,635 0 comments 0 reactions 1 assignee Claimed by @nsivabalan View on GitHub
area:sql from-jira priority:critical type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Currently, `MergeIntoHoodieTableCommand` squarely relies on semantic implemented by `ExpressionPayload` to be able to insert/update records.

While this is necessary since MIT semantic enables users to do sophisticated and fine-grained updates (for ex, partial updating), this is not necessary in the most generic case:

 
{code:java}
MERGE INTO target
USING ... source
ON target.id = source.id
WHEN MATCHED THEN UPDATE *
WHEN NOT MATCHED THEN INSERT *{code}
This is essentially just a SQL way of implementing an upsert – if there are matching records in the table we update them, otherwise – insert.

In this case there's actually no need to use ExpressionPayload at all, and we can just simply use normal Hudi upserting flow to handle it (avoiding all of the ExpressionPayload overhead)

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-5405
- Type: Bug
- Epic: https://issues.apache.org/jira/browse/HUDI-3249
- Affects version(s):
- 0.12.1
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.