apache / apache/doris

[Improvement] Avoid duplicate connector partition-pruning calls

Open
#67,739 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

## Background

Connector partition pruning can currently materialize a connector-native filtered partition view during logical pruning and then invoke connector filter pushdown again while preparing the physical scan handle. This may repeat the same remote metadata filter request.

## Goal

Introduce an explicit connector partition-pruning result that can carry the selected domain, updated connector handle or native metadata, consumed predicates, residual predicates, and an optional total partition count from logical planning into the physical scan.

## Scope

The result should avoid repeated remote pruning calls while preserving fallback behavior for unsupported connector dialects and keeping approximate or unsupported predicates as residual filters.

Contributor guide

Open the contributing guide

Research direction

Start by tracing connector partition pruning from logical planning to physical scan-handle preparation, focusing on where connector filter pushdown is invoked. Done means the pruning result carries the selected domain, handle or native metadata, consumed and residual predicates, and optional partition count while avoiding duplicate remote requests and preserving unsupported-dialect fallback behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.