Revisit HiveIncrPullSource and JdbcSource for interleaved inflight commits
Open
from-jira
priority:critical
type:bug
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
HUDI-5176
We have fixed the Hudi incremental source when there are inflight commits before completed commits. We need to revisit the logic for HiveIncrPullSource and JdbcSource as well regarding the same scenario.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5177
- Type: Bug
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the fix referenced in HUDI-5176, then compare how HiveIncrPullSource and JdbcSource handle interleaved inflight and completed commits. Done means both sources correctly handle the same scenario as the existing incremental source fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100