perf: cache `ArrayRef` in `RowGroupIndexReader` when row group doesn't change
Open
enhancement
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
- Related https://github.com/apache/arrow-rs/pull/9117
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
> From @alamb
>
> The other thing would be to pre-calculate the arrays (where the row group values don't change) and return the same ArrayRef until the RowGroup actually changes
Contributor guide
Research direction
Start by locating RowGroupIndexReader and reviewing the related pull request #9117 for context on the row-group values and ArrayRef handling. Done means returning the same pre-calculated ArrayRef while the row group is unchanged, and refreshing it when the RowGroup changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100