kestra-io / kestra-io/plugin-serdes
IonToParquet compression level with Zstd
- Dominant language
- Java
- Stars
- 1
- Forks
- 20
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 36
Description
### Feature description
In the serdes IonToParquet, we can choose the compression type (UNCOMPRESSED, SNAPPY, GZIP, ZSTD), but we can't choose the compression level for Zstd. I think without this, the default is 3, but it could be usefull to be able to change this value (ex: store cold result with 22 compression level)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the IonToParquet entry point and trace how the selected compression type is passed to the Parquet writer. Check how ZSTD currently uses its default level and where an additional compression-level option could be exposed. Done means callers can choose the Zstd level, including values such as 22, without changing the existing behavior for other compression types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100