Add ordering value to RLI partition
Open
area:metadata-table
area:writer
from-jira
priority:high
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
For now, when we add records to log files in data table, to generate RLI records, we avoid reading older log files/entire file slice to account for deletes w/ lower ordering value.
One way we could solve this is to store the ordering value in RLI partition. And so, while reading records from log file, while merging w/ previous value in RLI we can use that to determine the final resolved state.
this means, we have to upgrade the schema for RLI.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9341
- Type: Improvement
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.