[QST] Invitation to provide feedback on New Parquet Encoding: Adaptive Lossless Floating Point Compression
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
👋 from the Parquet community
The Parquet community is working on a proposal to implement [ALP: Adaptive Lossless floating-Point Compression, (SIGMOD 2024)](https://dl.acm.org/doi/10.1145/3626717) as a new Parquet encoding
cuDF is listed on the [Parquet Implementation Status](https://parquet.apache.org/docs/file-format/implementationstatus/) page
Since you may feel pressure to implement newly added encodings, I think having your feedback on the spec as a stakeholder would be immensely valuable.
Also, I believe you can help us understand if the encoding can be efficiently implemented on GPUs, which is an important consideration
The proposed addition is here : [Parquet ALP Encoding Specification](https://docs.google.com/document/d/1xz2cudDpN2Y1ImFcTXh15s-3fPtD_aWt/edit) (Google Doc)
There is also a prototype C++ implementation for review here
- https://github.com/apache/arrow/pull/48345
See also: [mailing list discussion (link)](https://lists.apache.org/thread/tjtln1mmjqfoql1ls2dw9xpdk91r1909) and [parquet-format PR](https://github.com/apache/parquet-format/pull/548)
Thank you for your consideration
Contributor guide
Assessment
This issue has not been assessed yet.