apache / apache/parquet-site

Write a Parquet Blog on new encoding Adaptive Lossless Floating Point (ALP)

Open
#175 2 comments 2 reactions 2 assignees Claimed by @sdf-jkl View on GitHub
Dominant language
HTML
Stars
17
Forks
55
Avg merge
1h 25m
Merged PRs (30d)
1

Description

- Part of https://github.com/apache/parquet-site/issues/146
- Related to https://github.com/apache/parquet-format/issues/533

We are in the process of adding ALP to the Parquet specification, a new encoding for floating point integers.

The change to the parquet-format specification from @prtkgaur is going well,
- https://github.com/apache/parquet-format/pull/557

However, the spec largely focuses on the low level technical details and it will be somewhat awkward to refer to to help people understand what ALP, when they should use it, and why they should add it to their systems. Therefore I think it would help to add a high level overview / link to point people at.

There is a lot of this content in the Google Doc draft ALP Proposal : [ALP : Floating point encoding for Parquet](https://docs.google.com/document/d/1PlyUSfqCqPVwNt8XA-CfRqsbc0NKRG0Kk1FigEm3JOg/edit?tab=t.0#heading=h.5xf60mx6q7xk) but it is intermixed with low level technical implementation details and commentary.

Thus, I suggest a blog post on https://parquet.apache.org/blog/ with a high level
1. Explains the usecase of ALP (decimal data that was stored in FLOAT/DOUBLE columns), single row decode
2. References the ALP paper (and probably mentions its adoption by other formats like Vortex)
2. Gives a technical overview of the encoding (with diagrams)
3. Gives some more examples

I also would like to have this post to continue the a story arc that highlights major new addition we are making to Parquet spec, how we are adopting cutting edge research, and and how they are being adopted (e.g. https://parquet.apache.org/docs/file-format/implementationstatus/)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.