[Enhancement] Selectivity-aware conjunct execution for ORC/Parquet readers (opt-in)
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues) and found no similar issues.
### Description
The ORC and Parquet readers evaluate every filter conjunct over all rows of a block and AND the results together, regardless of how cheap/selective each predicate is or the order the planner emitted them. An expensive predicate (e.g. a string function like `split_by_string(col)[n]='x'`) is therefore run over the full block even when a cheap, highly-selective predicate on the same block would have filtered almost everything first.
### Proposal
Add three **opt-in (default off)** session variables that only change the order/method of filter execution, never which rows pass:
1. `enable_scan_conjunct_reorder` — static cost-based conjunct reorder (cheap predicates first).
2. `enable_scan_selective_filter` — selection-vector execution: a later predicate is evaluated only on the rows earlier ones let through, producing the identical full-width `result_filter`/`can_filter_all` contract.
3. `enable_scan_adaptive_reorder` — runtime reorder by measured cost/selectivity, correcting the static estimate for skewed data or unknown UDF cost.
All default **off**, so with the flags unset the reader path is byte-identical to today. When enabled, they change execution order/method only — results are identical.
### Use case
Faster ORC/Parquet scans on queries mixing cheap selective predicates with expensive ones (e.g. string/UDF predicates).
### Related issues
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start by tracing the ORC and Parquet reader paths that evaluate filter conjuncts, then inspect how the three session variables are defined and consumed. Verify that each flag is default-off, preserves the full-width result_filter/can_filter_all contract, and returns identical rows while improving execution for selective and expensive predicates.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100