Switch to official AWS SDK for S3 to support IMDSv2
- Dominant language
- Rust
- Stars
- 20.8k
- Forks
- 2.1k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 181
Description
**Is your feature request related to a problem? Please describe.**
The dependency used for connecting to S3 in cube.js is https://github.com/durch/rust-s3. It only supports IMDSv1 which is insecure and sometimes banned globally for some organisation. IMDSv2 is the secure replacement and isn't supported by `rust-s3`. As a result, cube.js running in AWS on EC2 (the problem may extend to anything AWS) is unable to access S3 and compute pre-aggregations using S3 since credentials can't be obtained from the instance via IMDS.
More documentation on IMDS at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html
**Describe the solution you'd like**
AWS has released their official rust SDK which includes S3 and is in developer preview. It would be great if we could add a feature/config flag to use the AWS SDK over `rust-s3` in cube.js. At least while it's in developer preview and ultimately switch to the AWS SDK completely
**Describe alternatives you've considered**
The alternative would be to stick with v1 which might be possible at some organisation. Using other authentication mechanisms such as keypairs is often banned as well.
**Additional context**
Code from `rust-s3` getting the credentials from instance metadata is [here](https://github.com/durch/rust-s3/blob/d54bbaabfdc971f745874f5cbb28080762810821/aws-creds/src/credentials.rs#L307) and you can see it's not requesting a token before issuing the request as recommended with v2
Contributor guide
Assessment
This issue has not been assessed yet.