Extend dictionary encoding options to table level
Open
enhancement
parquet-module
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 19h 46m
- Merged PRs (30d)
- 141
Description
Issue #670 allowed control of whether Parquet files are written with dictionary encoding. This was done at the instance level with separate properties for row keys, sort keys and value fields. This issue is to extend this to the table level, so that this can be controlled separately for each field in each table. It's not clear what the best way to express this in a property is, `sleeper.table.parquet.dictionary.enabled.fields=mykeyfield,myvaluefield`? The default should be for dictionary encoding not to be used.
Contributor guide
Assessment
This issue has not been assessed yet.