apache / apache/arrow-rs

Filter pushdown selectivity threshold

Open
#9,591 2 comments 0 reactions 0 assignees View on GitHub
parquet performance
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

## Summary

Skip pushing down row selection filters when the selection has low effectivity (selects nearly all rows), as the overhead of individual small skip/read calls during Parquet decoding can outweigh the benefit.

Related PR: #9414

Contributor guide

Open the contributing guide

Research direction

Start by reviewing related PR #9414 and tracing the Parquet filter pushdown and decoding path. Identify how selection effectivity is measured and where a selectivity threshold can be applied; done means highly inclusive selections avoid costly individual skip/read calls without losing filtering benefits.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.