Make merger and payload class be consisent
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
{{`HoodieRecordUtil.createRecordMerger`}} falls back to `{{{}HoodieAvroRecordMerger`{}}} without considering merger mode. However, `{{{}HoodieTableConfig.inferCorrectMergingBehavior`{}}} behaviors differently. That is, if the merge strategy id is given, it could use `{{{}Commit time based`{}}} payload class, which may contradict with the merger selection logic of `{{{}HoodieRecordUtil`{}}} . What is why sometimes we see there are inconsistent merging behavior. Since the behavior of `{{{}HoodieRecordUtil.createRecordMerger`{}}} affects MDT behavior, we need to update the `{{{}HoodieTableConfig.inferCorrectMergingBehavior`{}}} accordingly.
This issue may not impact broadly since this is a corner case.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8876
- Type: Bug
- Affects version(s):
- 1.2.0
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing HoodieRecordUtil.createRecordMerger with HoodieTableConfig.inferCorrectMergingBehavior, focusing on the fallback and merge strategy id described in the issue. Trace how the selected merger and payload class affect MDT behavior; done means both entry points select consistent merging behavior for the cited corner case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100