apache / apache/datafusion

CLI utility does not check expiration of AWS credentials

Open
#23,321 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

When fetching credentials for S3 object store:
https://github.com/apache/datafusion/blob/96a6096c6f4b924e8cab4bc1629759a948e12939/datafusion-cli/src/object_storage.rs#L240

Wrapper does not check [expiration](https://docs.rs/aws-credential-types/1.2.14/aws_credential_types/struct.Credentials.html#method.expiry) to cache these credentials which will result in every request to use potentially heavy provider operation

### To Reproduce

_No response_

### Expected behavior

_No response_

### Additional context

This is a not big deal as I believe majority of credentials will be static.
Nevertheless addition of `sso` feature on `aws-config` would result in many HTTP requests with such implementation (potentially leading to rate limit errors)
I'm not sure if other sources of auth will result in HTTP request spam
So JFYI

Contributor guide

Open the contributing guide

Research direction

Start in datafusion-cli/src/object_storage.rs around the credential-fetching code linked in the issue, then read the aws_credential_types Credentials expiry documentation. Trace how credentials are cached and reused, and verify that expiration-aware caching avoids repeatedly invoking the provider while still refreshing expired credentials.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
cli, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.