[filesystem] Evaluate DefaultCredentialsProvider for S3 authentication
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Description
PR #3699 migrates the Fluss S3 filesystem plugin from AWS SDK v1 to AWS SDK v2. The current Glue guide documents separate `fs.s3a.aws.credentials.provider` lists for ECS Fargate, EKS with IRSA, and EC2.
As a follow-up to the review discussion in #3699, evaluate whether the guide can instead recommend the AWS SDK v2 `software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider` in `server.yaml`. This would let the AWS SDK own environment detection while still allowing users to override the provider for specialized setups.
### Verification
- [ ] Confirm that Hadoop S3A 3.4.3 can load `DefaultCredentialsProvider` through `fs.s3a.aws.credentials.provider`.
- [ ] Manually verify the configuration with ECS task-role credentials.
- [ ] Manually verify the configuration with EKS IRSA credentials.
- [ ] Manually verify the configuration with an EC2 instance profile.
- [ ] Verify both server-side `remote.data.dir` access and the relevant Fluss filesystem security-token/client access path.
- [ ] Check whether provider fallback introduces noticeable cold-start delays.
- [ ] If the verification succeeds, simplify the Glue guide and document how users can configure a specific provider when needed.
Related discussion: https://github.com/apache/fluss/pull/3699#issuecomment-5227873480
Related PR: #3699
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Glue guide and server.yaml, then check whether Hadoop S3A 3.4.3 loads software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider through fs.s3a.aws.credentials.provider. Verify ECS task-role, EKS IRSA, and EC2 instance-profile access for remote.data.dir and the relevant Fluss security-token/client path. Done means the approach is validated, fallback delays are understood, and the guide documents both the default and provider override configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- authentication, cloud, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100