[C++][Parquet] Add Skip on Parquet decoders
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Add `Skip` on `TypedDecoder` in Parquet encoding.
https://github.com/apache/arrow/blob/3b515ede5fa270bab8ebbaba9afef816854a3708/cpp/src/parquet/encoding.h#L267
The absence of this function means that downstream users have to allocate and and decode into a scratch buffer to be able to throw away values.
While not all decoder can have an efficient `Skip` implementation, some can, and the remaining can adapt the value of a stack buffer to a size appropriate for their own `Decode` function.
The decoding in a scratch buffer is currently being privately implemented in a temporary wrapper around the decoder in
https://github.com/apache/arrow/blob/cc90e40e5703f13ef1e38dd728eda93961d336d6/cpp/src/parquet/column_reader.cc#L672
### Component(s)
C++, Parquet
Contributor guide
Research direction
Start in cpp/src/parquet/encoding.h at TypedDecoder::Skip and compare the decoder implementations that would need the API. Read the temporary scratch-buffer wrapper in cpp/src/parquet/column_reader.cc around line 672 to understand the current behavior. Done means decoders can discard values through the new operation without requiring downstream callers to allocate a decode buffer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100