apache / apache/hudi

Fail metadata table validation early for mismatch file slice if timeline has no inflight instant

Open
#15,113 0 comments 0 reactions 0 assignees View on GitHub
from-jira priority:medium type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

https://github.com/apache/hudi/pull/5234/files/700f80ec372c2a75cf75754f68d6ee2eb0e7fe3b#diff-67533f5d7bf0e672db06b465b914e313cd197ef9a1648f663e1da625df753eac

We can check data table timeline and check if there are any inflights. and if its committed in MDT and then proceed w/ further checks.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-3822
- Type: Task

---

## Comments

24/Jul/22 22:54;xushiyan;[~guoyihua] not sure if this is resolved. can you confirm pls?;;;

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked pull-request diff and the metadata-table validation path, then trace how the data table timeline exposes inflight instants and how commits are represented in the MDT. Done means a mismatch file slice is rejected early when there is no inflight instant, while the existing further checks still run for valid cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.