apache / apache/hudi

Fixed all follow ups from HUDI-9030

Open
#16,906 2 comments 0 reactions 0 assignees View on GitHub
area:writer from-jira priority:high status:pr-available type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Filing a tracking ticket for all follow ups from HUDI-9030

 

[https://github.com/apache/hudi/pull/12888/files#r1985816074]
[https://github.com/apache/hudi/pull/12888/files#r1985858219]
[https://github.com/apache/hudi/pull/12888/files#r1985859138]
Any changes to LogRecordScanner classes. not really required to be fixed right away. 

2. Listing based rollback strategy. 
3. Add tests to restore to a commit w/ long history w/ a mix of DC, compaction, clustering N no of times. all inline table service would do. But restore should succeed and data validation should remain intact. 
4. Fix the IOType as Append for log files with table version 6
5. We will need to check if configs addded in version 1.0 are required for tbl version 6
6. Ensure these are accounted for in FGR [https://github.com/apache/hudi/pull/12888/files#r1976856670] 

 

 

WIP patch: 

[https://github.com/nsivabalan/hudi/tree/fixTableVersion6FixesAbstraction]

 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9140
- Type: Improvement

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the HUDI-9030 follow-up discussion and the referenced files and comments in PR #12888, then inspect the LogRecordScanner classes and the WIP branch fixTableVersion6FixesAbstraction. Resolve the listed rollback, restore-history, log-file IOType, table-version configuration, and FGR concerns, with tests proving restore and data validation across mixed table services.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, distributed-systems, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.