[Improvement] Avoid duplicate connector partition-pruning calls
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
## Background
Connector partition pruning can currently materialize a connector-native filtered partition view during logical pruning and then invoke connector filter pushdown again while preparing the physical scan handle. This may repeat the same remote metadata filter request.
## Goal
Introduce an explicit connector partition-pruning result that can carry the selected domain, updated connector handle or native metadata, consumed predicates, residual predicates, and an optional total partition count from logical planning into the physical scan.
## Scope
The result should avoid repeated remote pruning calls while preserving fallback behavior for unsupported connector dialects and keeping approximate or unsupported predicates as residual filters.
Contributor guide
Research direction
Start by tracing connector partition pruning from logical planning to physical scan-handle preparation, focusing on where connector filter pushdown is invoked. Done means the pruning result carries the selected domain, handle or native metadata, consumed and residual predicates, and optional partition count while avoiding duplicate remote requests and preserving unsupported-dialect fallback behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100