CLI utility does not check expiration of AWS credentials
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
When fetching credentials for S3 object store:
https://github.com/apache/datafusion/blob/96a6096c6f4b924e8cab4bc1629759a948e12939/datafusion-cli/src/object_storage.rs#L240
Wrapper does not check [expiration](https://docs.rs/aws-credential-types/1.2.14/aws_credential_types/struct.Credentials.html#method.expiry) to cache these credentials which will result in every request to use potentially heavy provider operation
### To Reproduce
_No response_
### Expected behavior
_No response_
### Additional context
This is a not big deal as I believe majority of credentials will be static.
Nevertheless addition of `sso` feature on `aws-config` would result in many HTTP requests with such implementation (potentially leading to rate limit errors)
I'm not sure if other sources of auth will result in HTTP request spam
So JFYI
Contributor guide
Research direction
Start in datafusion-cli/src/object_storage.rs around the credential-fetching code linked in the issue, then read the aws_credential_types Credentials expiry documentation. Trace how credentials are cached and reused, and verify that expiration-aware caching avoids repeatedly invoking the provider while still refreshing expired credentials.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100