cpp: read_file_slice{,_from_paths} ignore ReadOptions (no projection/filter pushdown)
- Dominant language
- Rust
- Stars
- 279
- Forks
- 67
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 40
Description
## Summary
The C++ FFI's slice-read entry points in `cpp/src/lib.rs` — `read_file_slice` and `read_file_slice_from_paths` — construct `ReadOptions::new()` internally and pass that to the inner reader. As a result, any read options the caller might want (column projection, filters, read-optimized mode, batch size) never reach the read.
The Rust core already supports these: `FileGroupReader::read_file_slice_from_paths` honors `options.projection` and filters and applies them to the merged result. Only the C++ shim drops them.
## Impact for engine integrations
Engines embedding the C++ library (e.g. Presto native / Velox — see apache/hudi#18308) get no column pruning or predicate pushdown through the FFI. The merged batch materializes all columns for the slice even when the query needs a few, which is both slower and heavier on memory (a MOR slice with logs is merged in memory and returned as a single batch).
## Proposal
Widen the C++ slice-read entry points to accept read options and forward them to the inner reader. A simple, `cxx`-friendly shape is a `Vec` of `key=value` options (consistent with `new_file_group_reader_with_options`), with a reserved `projection=col1,col2` key mapped to `ReadOptions::with_projection`; other keys map to `with_hudi_option`. A dedicated projection parameter would also work.
This enables column projection immediately and leaves a clear path to filter pushdown later, without further FFI signature changes.
## Notes
Pairs with the panic-safety hardening tracked separately. Both are small, isolated changes to `cpp/src/lib.rs` and are needed for a usable query-engine integration.
Contributor guide
Assessment
This issue has not been assessed yet.