Fix callers of HoodieSparkUtils.createRdd() for last arg to leverage latest table schema
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
[https://github.com/apache/hudi/pull/2927] is adding support for latest table schema in HoodieSparkUtils.createRdd(). two of the callers are not fixed as part of the PR. Revisit and fix if required.
SparkParquetBootstrapDataProvider
SourceFormatAdaptor
SparkBasedReader
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-1907
- Type: Task
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing HoodieSparkUtils.createRdd() and the change in PR 2927 that adds latest table schema support. Then inspect SparkParquetBootstrapDataProvider, SourceFormatAdaptor, and SparkBasedReader to determine whether their final arguments use the updated schema behavior. The work is done when any necessary callers are updated consistently and the relevant Spark tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100