[C++][Parquet] Improve parquet reading performance for String/Binary type based on Buffer operations instead of BinaryArrayBuilder
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
Improve parquet reading performance for String/Binary type based on Buffer operations instead of BinaryArrayBuilder.
Currently, parquet String/Binary type reading performance is not good enough. For fixedwidth types, the Reader has adopted Buffer operations instead of ArrayBuilder interfaces.
So it‘s better to use Buffer operations in String/Binary type reading.
**Reporter**: [xiangxiang Shen](https://issues.apache.org/jira/browse/ARROW-17735) / @zhixingheyi-tian
**Assignee**: [xiangxiang Shen](https://issues.apache.org/jira/browse/ARROW-17735) / @zhixingheyi-tian
#### PRs and other links:
- [GitHub Pull Request #14353](https://github.com/apache/arrow/pull/14353)
**Note**: *This issue was originally created as [ARROW-17735](https://issues.apache.org/jira/browse/ARROW-17735). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reviewing the Parquet String/Binary reading path and its current BinaryArrayBuilder use, then compare it with the existing fixed-width Buffer operations. The work is complete when String/Binary reads use the intended Buffer-based approach and the reported reading-performance problem is addressed; pull request #14353 is already linked in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 15/100