apache / apache/parquet-format
Support gorilla encoding for float numbers
- Dominant language
- Thrift
- Stars
- 2.6k
- Forks
- 508
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
[Gorilla](https://www.vldb.org/pvldb/vol8/p1816-teller.pdf) is a de facto encoding algorithm for float numbers, it has been used by many time series database such as InfluxDB, TimescaleDB for a while.
For now Parquet only supports BYTE_STREAM_SPLIT encoding for float numbers, it would be nice to add gorilla encoding.
**Reporter**: [Frank Dai](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=soulmachine)
**Note**: *This issue was originally created as [PARQUET-2217](https://issues.apache.org/jira/browse/PARQUET-2217). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
Research direction
Start by reviewing Parquet's encoding specification and the existing BYTE_STREAM_SPLIT encoding support, then determine the format changes required for Gorilla encoding. Done means Gorilla encoding for floating-point numbers is specified and supported consistently across the format; the issue provides no file or test entry point.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100