Support Nested Schema pruning optimization when Schema Evolution is on
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Currently, in cases when Comprehensive Schema evolution is enabled `NestedSchemaPruning` optimization will not be applied.
Conceptually there's no reason for that and we should be able to apply it even in that case.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5421
- Type: Bug
- Epic: https://issues.apache.org/jira/browse/HUDI-2429
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the NestedSchemaPruning implementation and the code path for Comprehensive Schema evolution in the Hudi codebase. Trace why pruning is currently skipped, then verify that it is applied with schema evolution enabled and add or update regression coverage for that combination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100