Support bootstrap files in file group reader-based compaction
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
testMetadataBootstrapMORPartitionedInlineCompactionOn fails with file group reader-based compaction (HoodieSparkMergeHandleV2). Currently if the compaction plan and operations contain bootstrap files, the compaction goes through the old flow using the regular HoodieMergeHandle. We should support bootstrap files in file group reader-based compaction in HoodieSparkMergeHandleV2.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8632
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-9107
- Fix version(s):
- 1.1.0
---
## Comments
22/Jan/25 05:52;yihua;Found another issue: HUDI-8896. Right now reading partition columns for bootstrapped table (bootstrap skeleton + data file) relies on engine-specific handling, not in the file group reader. So compaction and clustering using the file group reader to read out records from metadata-only bootstrapped table can encounter nulls in the partition columns, causing the new base file written to have nulls in the partition columns.;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.