[C++][Parquet] Optimize DELTA_BINARY_PACKED encoding and decoding
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
[As suggested here](https://github.com/apache/arrow/pull/14191#discussion_r1019762308) simd approach such as [FastDifferentialCoding](https://github.com/lemire/FastDifferentialCoding) could be used to speed up encoding and decoding.
**Reporter**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-18365) / @rok
**Watchers**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-18365) / @rok
**Note**: *This issue was originally created as [ARROW-18365](https://issues.apache.org/jira/browse/ARROW-18365). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by locating the Parquet DELTA_BINARY_PACKED encoding and decoding implementations and measuring their current performance. Review the FastDifferentialCoding approach linked in the issue, then define the optimization scope and benchmarks needed to show that both paths are faster without changing their behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100