cube-js / cube-js/cube

Cubestore - ADD Snowflake Storage Integration to Access Amazon S3

Open
#5,472 1 comment 0 reactions 0 assignees View on GitHub
driver:snowflake help wanted
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Is your feature request related to a problem? Please describe.**
This is [barely documented at all](https://cube.dev/docs/config/databases/snowflake), but it seems Snowflake driver's S3 `Export Bucket` functionality currently works by:
1. Creating temp pre-agg tables in Snowflake (with `CREATE` commands)
2. Using a Snowflake `COPY` command to move this temp data into the S3 bucket you've specified in the CUBEJS_EXPORT_ and CUBESTORE env variables, including the access key/secret.

This is not the ideal way to integrate Snowflake with S3. Snowflake has a concept of an [S3 Storage Integration](https://docs.snowflake.com/en/user-guide/data-load-s3-config-storage-integration.html).

**Describe the solution you'd like**
Users should be allowed to set this up and specify their integration, rather than need to pass the access key/secret from Cube to Snowflake. There are many preferred ways to access an S3 bucket from a certain place (for example, EKS where Cube may be running), and the underlying Rust-AWS-auth lib seems to already support them.

Matter of fact, [this Snowflake integration seems to have already been added for GCP buckets](https://cube.dev/docs/config/databases/snowflake#pre-aggregation-build-strategies).

**Describe alternatives you've considered**
No workarounds that are secure.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.