[Parquet] Validate the ALP sampling heuristic
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
### Is your feature request related to a problem or challenge?
The ALP encoder samples input values to select candidate exponent and factor pairs. We should verify that this heuristic produces compression reasonably close to evaluating every valid pair.
Pathological inputs where sampling selects significantly worse parameters should become deterministic regression cases.
This depends on #9372 and should be worked on after that PR is merged.
Contributor guide
Research direction
After #9372 is merged, locate the ALP encoder and its sampling heuristic. Compare sampled candidate selection with evaluation of every valid exponent and factor pair, then identify pathological inputs where the result is significantly worse. Add deterministic regression cases demonstrating the expected compression-quality behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, performance, testing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100