gchq / gchq/sleeper

Extend dictionary encoding options to table level

Open
#671 0 comments 0 reactions 0 assignees View on GitHub
enhancement parquet-module
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

Issue #670 allowed control of whether Parquet files are written with dictionary encoding. This was done at the instance level with separate properties for row keys, sort keys and value fields. This issue is to extend this to the table level, so that this can be controlled separately for each field in each table. It's not clear what the best way to express this in a property is, `sleeper.table.parquet.dictionary.enabled.fields=mykeyfield,myvaluefield`? The default should be for dictionary encoding not to be used.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.