apache / apache/hudi

Handle inconsistent views of source table and Hudi dataset

Open
#14,444 0 comments 0 reactions 0 assignees View on GitHub
from-jira priority:blocker type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

As mentioned in the RFC, there are certain scenarios in which the source table and hudi dataset are inconsistent with each other. There were some suggestions during the discussion of [RFC|https://cwiki.apache.org/confluence/display/HUDI/RFC+-+14+%3A+JDBC+incremental+puller#RFC14:JDBCincrementalpuller-Differentmodesoffetching]. We need to evaluate them.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-1862
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-1859

---

## Comments

04/Oct/21 13:58;vinoth;[~codope] [~rmahindra] would like to declare the JDBC source GA this release.
lets use this task to generally fix anything that needs to be done before tht.;;;

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading RFC 14, especially the section on different JDBC incremental-puller fetching modes, and review the linked HUDI-1862 and HUDI-1859 context. Evaluate the suggested approaches for source-table and Hudi-dataset inconsistency, then define and verify a resolution suitable for declaring the JDBC source GA.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.