Refactor hudi-spark integration
Open
from-jira
priority:high
type:bug
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Refactor spark integration to get rid of common modules/classes shared across Spark versions. Instead create new copies for each spark version
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6429
- Type: Bug
- Epic: https://issues.apache.org/jira/browse/HUDI-6427
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the Hudi Spark integration modules and the shared classes used across Spark versions. Compare the version-specific integration paths and identify the common modules that must be replaced. Done means each Spark version has its own implementation and the shared Spark-version modules and classes are removed, with the integration tests passing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100