[Parquet] Support configurable ALP vector lengths
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
### Is your feature request related to a problem or challenge?
The Parquet ALP specification permits `log_vector_size` values from 3 through 15, but arrow-rs currently writes only the recommended default size of 1024 values.
This prevents testing or selecting other vector sizes permitted by the format.
This depends on #9372 and should be worked on after that PR is merged.
### Describe the solution you would like
Add an ALP writer configuration for selecting any vector size permitted by the Parquet specification.
The existing default should remain 1024 values.
Contributor guide
Research direction
Review dependency #9372 first, then inspect the existing ALP writer configuration and how it uses the default vector size. Confirm the writer accepts the Parquet-permitted log_vector_size range of 3 through 15, preserves the 1024-value default, and supports selecting another permitted size.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100