NVIDIA / NVIDIA/cudf

[QST] Invitation to provide feedback on New Parquet Encoding: Adaptive Lossless Floating Point Compression

Open
#21,173 9 comments 5 reactions 0 assignees View on GitHub
cuIO libcudf question
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

👋 from the Parquet community

The Parquet community is working on a proposal to implement [ALP: Adaptive Lossless floating-Point Compression, (SIGMOD 2024)](https://dl.acm.org/doi/10.1145/3626717) as a new Parquet encoding

cuDF is listed on the [Parquet Implementation Status](https://parquet.apache.org/docs/file-format/implementationstatus/) page

Image

Since you may feel pressure to implement newly added encodings, I think having your feedback on the spec as a stakeholder would be immensely valuable.

Also, I believe you can help us understand if the encoding can be efficiently implemented on GPUs, which is an important consideration

The proposed addition is here : [Parquet ALP Encoding Specification](https://docs.google.com/document/d/1xz2cudDpN2Y1ImFcTXh15s-3fPtD_aWt/edit) (Google Doc)

There is also a prototype C++ implementation for review here
- https://github.com/apache/arrow/pull/48345

See also: [mailing list discussion (link)](https://lists.apache.org/thread/tjtln1mmjqfoql1ls2dw9xpdk91r1909) and [parquet-format PR](https://github.com/apache/parquet-format/pull/548)

Thank you for your consideration

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.