Batch snapshot reads ignore hoodie.file.group.reader.enabled
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Since #14061, `hoodie.file.group.reader.enabled` is only consulted by the streaming sources (`HoodieStreamSourceV1`/`V2`) and `HoodieIncrSource`. `DefaultSource` routes every batch snapshot and read-optimized query to `HoodieFileGroupReaderBasedFileFormat` regardless of the config, so there is no runtime way to run the same query against the legacy read path.
That gap shows up when a read regression is suspected on the file group reader. On 1.1.x this config was the switch used to compare the two read paths on the same table and cluster; on 1.2.0 and master there is no equivalent, and the config documentation ("Use engine agnostic file group reader if enabled") still reads as if it applies to batch reads.
The legacy relations for snapshot queries are all still in the tree (`BaseFileOnlyRelation`, `MergeOnReadSnapshotRelation`) and still serve metadata-table reads and streaming reads with the config off, so honoring the config for batch snapshot queries again is a routing change only.
Proposal: have `DefaultSource` consult `hoodie.file.group.reader.enabled` for COW snapshot/read-optimized and MOR snapshot queries, leaving the default (config on) behavior unchanged. CDC, incremental and bootstrap queries stay on the file group reader unconditionally, since their legacy relations have been removed.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.