Fail metadata table validation early for mismatch file slice if timeline has no inflight instant
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
https://github.com/apache/hudi/pull/5234/files/700f80ec372c2a75cf75754f68d6ee2eb0e7fe3b#diff-67533f5d7bf0e672db06b465b914e313cd197ef9a1648f663e1da625df753eac
We can check data table timeline and check if there are any inflights. and if its committed in MDT and then proceed w/ further checks.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3822
- Type: Task
---
## Comments
24/Jul/22 22:54;xushiyan;[~guoyihua] not sure if this is resolved. can you confirm pls?;;;
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked pull-request diff and the metadata-table validation path, then trace how the data table timeline exposes inflight instants and how commits are represented in the MDT. Done means a mismatch file slice is rejected early when there is no inflight instant, while the existing further checks still run for valid cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100