Parquet Statistics Pruning Ignores ColumnOrder, resulting in potentially incorrect statistics
Open
bug
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
The statistics are only valid if interpreted in the context of [ColumnOrder](https://docs.rs/parquet/latest/parquet/basic/enum.ColumnOrder.html), otherwise the results are not necessarily correct
The `ColumnOrder` field in Parquet statistics says what ordering was used to compute the min/max values and seems not to be widely used or populated in the eco system. However, ignoring it when it is present is probably wrong
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.