cube-js / cube-js/cube

Switch to official AWS SDK for S3 to support IMDSv2

Open
#4,041 1 comment 4 reactions 0 assignees View on GitHub
help wanted
Dominant language
Rust
Stars
20.8k
Forks
2.1k
Avg merge
1d 2h
Merged PRs (30d)
181

Description

**Is your feature request related to a problem? Please describe.**
The dependency used for connecting to S3 in cube.js is https://github.com/durch/rust-s3. It only supports IMDSv1 which is insecure and sometimes banned globally for some organisation. IMDSv2 is the secure replacement and isn't supported by `rust-s3`. As a result, cube.js running in AWS on EC2 (the problem may extend to anything AWS) is unable to access S3 and compute pre-aggregations using S3 since credentials can't be obtained from the instance via IMDS.
More documentation on IMDS at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html

**Describe the solution you'd like**
AWS has released their official rust SDK which includes S3 and is in developer preview. It would be great if we could add a feature/config flag to use the AWS SDK over `rust-s3` in cube.js. At least while it's in developer preview and ultimately switch to the AWS SDK completely

**Describe alternatives you've considered**
The alternative would be to stick with v1 which might be possible at some organisation. Using other authentication mechanisms such as keypairs is often banned as well.

**Additional context**
Code from `rust-s3` getting the credentials from instance metadata is [here](https://github.com/durch/rust-s3/blob/d54bbaabfdc971f745874f5cbb28080762810821/aws-creds/src/credentials.rs#L307) and you can see it's not requesting a token before issuing the request as recommended with v2

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.