Improve the RLE encoding for Parquet Dictionary IDs
- Vorherrschende Sprache
- Java
- Sterne
- 3.1k
- Forks
- 1.6k
- Ø Merge
- 3 T. 12 Std.
- Gemergte PRs (30 T.)
- 33
Beschreibung
The IDs of Parquet Dictionary encoding is using `RunLengthBitPackingHybridEncoder`.
RunLengthBitPackingHybridEncoder handles encoding with `repeat` and `bitpacking`, we should improve it with the method likes `DeltaBinaryPackingWriter`
**Reporter**: [Dapeng Sun](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=dapengsun) / @sundapeng
**Note**: *This issue was originally created as [PARQUET-1059](https://issues.apache.org/jira/browse/PARQUET-1059). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Start by locating RunLengthBitPackingHybridEncoder and DeltaBinaryPackingWriter in the Java sources, then read their encoding and test coverage. Determine the intended improved handling for Parquet Dictionary IDs and define completion through encoding correctness and performance tests; the issue does not name specific files or tests.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java
- Bereich
- data-engineering
- Issue-Typ
- Refactoring
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100