Filtered Parquet scans issue additional object-store requests after ParquetPushDecoder migration
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
## Describe the bug
After [PR #20839](https://github.com/apache/datafusion/pull/20839) replaced the pull-based Parquet reader with `ParquetPushDecoder`, a controlled comparison found substantially more object-store requests for a large filtered Parquet scan.
The leading explanation is that predicate and projected-column ranges are discovered in separate `NeedsData` rounds. DataFusion can coalesce ranges within one `get_byte_ranges` call, but not with ranges discovered in a later decoder round. Current `main` still follows this sequence:
```text
try_next_reader()
-> NeedsData(ranges)
-> get_byte_ranges(ranges).await
-> push_ranges(...)
-> repeat
```
In architectures where compute and storage are decoupled, each additional object-store round trip adds directly to scan latency. This is especially significant when scans touch many small Parquet files.
The request-level attribution to predicate versus projection rounds is inferred from the source and the A/B results below. The additional requests themselves are directly measured.
## To reproduce
Compare a selective filter with a narrow projection over many remote Parquet files while instrumenting `AsyncFileReader` request counts. Our controlled comparison upgraded only DataFusion from 53.1.0 to 54.1.0 while retaining `parquet` 58.3.0 and `object_store` 0.13.2:
| Configuration | Object-store requests | Bytes read |
|---|---:|---:|
| DataFusion 53.1.0 | 708K-709K | ~139 GB |
| DataFusion 54.1.0 with filter pushdown | 894K-898K | ~86.6 GB |
| DataFusion 54.1.0 without filter pushdown | 583K | ~186.3 GB |
The DF54 request increase closely matched one additional request per filtered file.
Disabling filter pushdown is not a viable workaround. It reduces request count but more than doubles bytes read and produced a 164-second runtime in this workload.
## Expected behavior
Retain the byte savings from predicate pushdown without requiring a separate network round trip for predicate and projected-column ranges.
Possible directions include:
- Coalesce ranges across consecutive decoder rounds.
- Speculatively fetch projected-column ranges where appropriate.
- Extend the arrow-rs decoder API to expose upcoming optional ranges.
- Add request-count coverage using a counting, delayed object store.
## Additional context
- PR #20839 explicitly identified coalescing and prefetching as goals of the push-decoder migration.
- Closed [PR #21370](https://github.com/apache/datafusion/pull/21370) coalesced adjacent ranges within one `NeedsData` result, but did not address ranges discovered across separate rounds.
- A search did not find an existing DataFusion issue covering this cross-round request amplification.
Contributor guide
Research direction
Start by tracing the ParquetPushDecoder flow from try_next_reader through NeedsData, get_byte_ranges, and push_ranges, using PRs #20839 and #21370 as context. Reproduce the request amplification with a counting, delayed object store and compare filtered scans. Done means predicate and projected-column ranges avoid unnecessary cross-round object-store requests while retaining predicate-pushdown byte savings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100