apache / apache/fluss

[File system] Allow users to configure STS endpoint for S3 file system

Open
#613 6 comments 0 reactions 0 assignees View on GitHub
component=filesystem
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.

### Motivation

Allow users to explicitly configure the [STS endpoint](https://docs.aws.amazon.com/general/latest/gr/sts.html) that should be used.

### Solution

- Add the following config options for the S3 file system

- `fs.s3a.sts.endpoint`: The endpoint URL of the AWS Security Token Service to use for authenticating to S3.
- `fs.s3a.sts.region`: The region to use for SigV4 signing of requests. If not set, `fs.s3a.region` will be used.

- If `fs.s3a.sts.endpoint` is set, build the STS client in `src/main/java/com/alibaba/fluss/fs/s3/token/S3DelegationTokenProvider.java` with the desired endpoint configuration.

### Anything else?

_No response_

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with src/main/java/com/alibaba/fluss/fs/s3/token/S3DelegationTokenProvider.java and trace how the S3 STS client is built. Add support for fs.s3a.sts.endpoint and fs.s3a.sts.region, including fallback to fs.s3a.region, and verify that configured values affect endpoint selection and SigV4 signing.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
backend, cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.