Add an iterator api for HoodieUnMergedLogRecordScanner
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
HoodieUnMergedLogRecordScanner extends from
AbstractHoodieLogRecordReader, the later is used for merging multiple log files within one file slice. While for HoodieUnMergedLogRecordScanner, there is no merging, all we need to do is resolving the hoodie record and return it back.
The current impl is kind of hacky, it allows the HoodieUnMergedLogRecordScanner to pass around a
LogRecordScannerCallback, each resolved record in applied with this callback, the singular callback impl in Flink side is just to put the record into another queue, and pop the records again with this queue.
Let's refactor this:
1. add a iterator API for HoodieUnMergedLogRecordScanner, there might need some code refactoring also to the AbstractHoodieLogRecordReader to make the code more de-coupled;
2. in Flink side, use this new iterator instead to decipher the records.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8163
- Type: Improvement
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.