[C++][Dataset] Add support for filter pushdown in the ORC Scanner
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
In arrow dataset, Filter pushdown can improve reading files performance greatly. We notice parquet has implemented, https://github.com/apache/arrow/blob/35b3567e73423420a99dbe6116f000e3c77d2a4c/cpp/src/arrow/dataset/file_parquet.cc#L465-L484.
But ORC fileformat has not supported Filter pushdown. It ignores the "filter" of ScanOptions now.
**Reporter**: [xiangxiang Shen](https://issues.apache.org/jira/browse/ARROW-14890) / @zhixingheyi-tian
**Note**: *This issue was originally created as [ARROW-14890](https://issues.apache.org/jira/browse/ARROW-14890). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by comparing the ORC scanner's handling of ScanOptions with the parquet implementation in cpp/src/arrow/dataset/file_parquet.cc around lines 465-484. Trace where the ORC scanner currently ignores the filter, then verify that ORC scans apply filter pushdown and preserve the expected filtered results and performance behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100