Remove Reliance on Relations in the IncrementalFileIndex
Open
engine:spark
from-jira
priority:high
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Right now, we use methods on an mor incremental relation in the IncrementalFileIndex. We should create shared util methods instead of using the relation. This will help us for when we get rid of the relations
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9623
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating IncrementalFileIndex and cataloging the methods it calls on an incremental relation. Trace those calls to determine the shared utility boundaries; done means IncrementalFileIndex no longer relies on the relation and the relevant behavior remains covered by the existing tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100