Fixed all follow ups from HUDI-9030
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Filing a tracking ticket for all follow ups from HUDI-9030
[https://github.com/apache/hudi/pull/12888/files#r1985816074]
[https://github.com/apache/hudi/pull/12888/files#r1985858219]
[https://github.com/apache/hudi/pull/12888/files#r1985859138]
Any changes to LogRecordScanner classes. not really required to be fixed right away.
2. Listing based rollback strategy.
3. Add tests to restore to a commit w/ long history w/ a mix of DC, compaction, clustering N no of times. all inline table service would do. But restore should succeed and data validation should remain intact.
4. Fix the IOType as Append for log files with table version 6
5. We will need to check if configs addded in version 1.0 are required for tbl version 6
6. Ensure these are accounted for in FGR [https://github.com/apache/hudi/pull/12888/files#r1976856670]
WIP patch:
[https://github.com/nsivabalan/hudi/tree/fixTableVersion6FixesAbstraction]
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9140
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the HUDI-9030 follow-up discussion and the referenced files and comments in PR #12888, then inspect the LogRecordScanner classes and the WIP branch fixTableVersion6FixesAbstraction. Resolve the listed rollback, restore-history, log-file IOType, table-version configuration, and FGR concerns, with tests proving restore and data validation across mixed table services.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100