apache / apache/parquet-format

The example in delta-encoding seems incorrect

Open
#393 0 comments 0 reactions 0 assignees View on GitHub
Priority: Minor Type: bug
Dominant language
Thrift
Stars
2.6k
Forks
508
Avg merge
2d 8h
Merged PRs (30d)
4

Description

In the example using delta-encoded, encoding [1, 2, 3, 4, 5], we state that

```java

The final encoded data is:

header: 8 (block size), 1 (miniblock count), 5 (value count), 1 (first value)

block 1 (minimum delta), 0 (bitwidth), (no data needed for bitwidth 0)
```

I believe that the correct result should be

header: [8, 1, 5, 2]
block: [2, 0]

I.e first_value and min_delta should be 2, not 1.

This is because the zig-zag ULEB128-encoding of 1 is 2: the ULEB-128 encoding of 1 is 1, but AFAIK the zig-zag encoding of 1 is 2 (see e.g. [here](https://stackoverflow.com/a/2211086/931303)).

Alternatively, we could re-phrase "The final encoded data is:" to "The final data prior to zig-zag encoding is:"

**Reporter**: [Jorge Leitão](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=jorgecarleitao) / @jorgecarleitao

**Note**: *This issue was originally created as [PARQUET-2028](https://issues.apache.org/jira/browse/PARQUET-2028). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

Open the contributing guide

Research direction

Search the documentation for the delta-encoding example containing “The final encoded data is” and compare its values with the issue’s zig-zag ULEB128 explanation. Done means the example is explicitly correct, either by showing the zig-zag-encoded values or by clearly identifying the data as prior to zig-zag encoding.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.