apache / apache/arrow-adbc

feat(go/adbc/driver/snowflake): Expose file compression options for bulk ingestion

Open
#1,473 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
627
Forks
217
Avg merge
17h
Merged PRs (30d)
57

Description

As part of the updates to snowflake bulk ingestion (#1456), parquet files are compressed with configurable codec and compression level. These parameters currently cannot be changed, but we would like to specify them using `Statement.SetOption()`.

Specifically, users should be able to set the following keys on the Snowflake `Statement`:
- `adbc.snowflake.rpc.ingest_compression_codec`
- `adbc.snowflake.rpc.ingest_compression_level`

It would be nice if a `string` -> `Codec` method could be used from the `parquet` library directly, so that the list of options on the `Statement` is automatically kept up to date with the list of supported codecs.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.