Handle inconsistent views of source table and Hudi dataset
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
As mentioned in the RFC, there are certain scenarios in which the source table and hudi dataset are inconsistent with each other. There were some suggestions during the discussion of [RFC|https://cwiki.apache.org/confluence/display/HUDI/RFC+-+14+%3A+JDBC+incremental+puller#RFC14:JDBCincrementalpuller-Differentmodesoffetching]. We need to evaluate them.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-1862
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-1859
---
## Comments
04/Oct/21 13:58;vinoth;[~codope] [~rmahindra] would like to declare the JDBC source GA this release.
lets use this task to generally fix anything that needs to be done before tht.;;;
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading RFC 14, especially the section on different JDBC incremental-puller fetching modes, and review the linked HUDI-1862 and HUDI-1859 context. Evaluate the suggested approaches for source-table and Hudi-dataset inconsistency, then define and verify a resolution suitable for declaring the JDBC source GA.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100