apache / apache/arrow-go

Invitation to provide feedback on New Parquet Encoding: Adaptive Lossless Floating Point Compression

Open
#646 5 comments 1 reaction 0 assignees View on GitHub
Type: enhancement
Dominant language
Assembly
Stars
404
Forks
145
Avg merge
2d 4h
Merged PRs (30d)
87

Description

### Describe the enhancement requested

👋 from the Parquet community

The Parquet community is working on a proposal to implement [ALP: Adaptive Lossless floating-Point Compression, (SIGMOD 2024)](https://dl.acm.org/doi/10.1145/3626717) as a new Parquet encoding

arrow-go is listed on the [Parquet Implementation Status](https://parquet.apache.org/docs/file-format/implementationstatus/) page

Image

Since you may feel pressure to implement newly added encodings, I think having your feedback on the spec as a stakeholder would be immensely valuable.

The proposed addition is here : [Parquet ALP Encoding Specification](https://docs.google.com/document/d/1xz2cudDpN2Y1ImFcTXh15s-3fPtD_aWt/edit) (Google Doc)

There is also a prototype C++ implementation for review here
- https://github.com/apache/arrow/pull/48345

See also: [mailing list discussion (link)](https://lists.apache.org/thread/tjtln1mmjqfoql1ls2dw9xpdk91r1909) and [parquet-format PR](https://github.com/apache/parquet-format/pull/548)

Thank you for your consideration

### Component(s)

Parquet

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.