apache / apache/datafusion

Implement the `contained` method of `RowGroupPruningStatistics` introduce by #8440

Open
#8,668 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Currently the `contained` method of `RowGroupPruningStatistics` is not fully implemented.

### Describe the solution you'd like

The basic idea is to check whether all of the `values` are not within the min-max boundary.
- If any one value is within the min-max boundary, then this row group will not be skipped.
- Otherwise, this row group will be able to be skipped.

### Describe alternatives you've considered

_No response_

### Additional context

#8440

Contributor guide

Open the contributing guide

Research direction

Start by locating the RowGroupPruningStatistics implementation introduced by #8440 and read its existing min-max pruning logic. Trace how contained is called, then verify the behavior for values inside and outside the boundaries with the relevant pruning tests or by adding focused coverage; done means row groups are skipped only when no value can be contained.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.