apache / apache/arrow

[C++][Parquet] Reject outlandish values in DELTA_BINARY_PACKED decoder

Open
#50,314 3 comments 0 reactions 0 assignees View on GitHub
Component: C++ Component: Parquet Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

In `DeltaBitPackDecoder::InitHeader` we're allocating scratch space for miniblock decoding. The size of this scratch space is entirely driven by values read from the [DELTA_BINARY_PACKED header](https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5), and could therefore be outlandishly huge.

Given that a huge miniblock size is unlikely to be seen in the wild (and would defeat the point of having miniblocks), we should probably reject non-reasonable values in `InitHeader`.

### Component(s)

C++, Parquet

Contributor guide

Open the contributing guide

Research direction

Locate the C++ implementation of DeltaBitPackDecoder::InitHeader and read how DELTA_BINARY_PACKED header values determine scratch-space allocation. Check the existing Parquet decoder tests, then add coverage showing that unreasonable miniblock sizes are rejected and that normal headers still decode successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.