[FEA] Decode Parquet decimals directly into prescribed per-column types
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Problem**
The Parquet reader decimal-width option applies one width to all decimal columns and currently casts after decoding. Consumers may need mixed target widths in one read, including nested decimal leaves, without the extra allocation and cast kernel.
**Request**
Allow callers to prescribe decimal types per column or nested leaf and decode directly into those target buffers in the regular, chunked, and hybrid readers.
Motivating use case: https://github.com/facebookincubator/velox/pull/19042
Related: #20062 and #21472.
Contributor guide
Research direction
Start by comparing the regular, chunked, and hybrid Parquet reader paths, then read related issues #20062 and #21472 and the motivating pull request. Done means callers can prescribe different decimal types for columns and nested leaves, with decoding written directly into those target buffers without a separate cast allocation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100