apache / apache/arrow

[C++][Parquet] Speed up DELTA_BINARY_PACKED decoding

Open
#51,268 0 comments 0 reactions 1 assignee Claimed by @prtkgaur View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

`DeltaBitPackDecoder` decodes noticeably slower than the other integer encodings, for two
reasons that are both in the decoder rather than the format:

1. The prefix-sum loop reloads the frame and stores the running value on every value, because
both members may alias the caller-owned output buffer.
2. The bit unpacker is called once per miniblock -- one call per 32 values with the default
geometry -- even though miniblocks sharing a bit width are packed contiguously and can be
unpacked in a single call.

Together these are 1.5x-1.7x on the DELTA_BINARY_PACKED decode benchmarks already in the tree,
on the arms where the running sum and per-call setup are a meaningful share of the work.

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.