Incremental read using v2 datasource
Open
engine:spark
from-jira
priority:high
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
currently, we still use v1 format for incremental read, and need to use v2 format as well.
see comment: https://github.com/apache/hudi/pull/4611#discussion_r795089099
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3416
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-1297
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the linked pull-request discussion and tracing the incremental-read path that currently uses the v1 datasource format. Identify where the v2 format must also be handled, then verify that incremental reads support both formats without breaking the existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100