[Feature][Connector-V2] Support multiple credential providers for OssFile connector
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
### Search before asking
- [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
Currently, the OssFile connector (both source and sink) only supports a single authentication mode: directly configuring `access_key` and `access_secret` in the job configuration file. This has significant security and usability limitations in cloud environments.
This feature proposes adding support for multiple OSS credential providers through a new configuration option `fs.oss.credentials.provider`, which accepts a list of provider class names and tries them in order (chain pattern) until one succeeds.
**Proposed Authentication Modes:**
| Mode | Provider Class | Description |
|------|---------------|-------------|
| AK/SK from config | (default, no provider needed) | Backward compatible, reads `access_key`/`access_secret` from config file |
| Environment variables | `EnvironmentVariableCredentialsProvider` | Reads `OSS_ACCESS_KEY_ID` / `OSS_ACCESS_KEY_SECRET` from environment variables |
| ECS RAM Role | `OssInstanceProfileCredentialsProvider` | Credential-free access via ECS metadata service, requires `role_name` |
| STS Token | `OssStsCredentialsProvider` | Temporary credentials with `access_key`, `access_secret` and `security_token` |
**New Configuration Options:**
| Option | Type | Required | Description |
|--------|------|----------|-------------|
| `fs.oss.credentials.provider` | `List` | No | Credential provider class names, tried in order |
| `role_name` | `String` | No | ECS RAM role name (for InstanceProfile provider) |
| `security_token` | `String` | No | STS security token (for STS provider) |
**Design Details:**
1. **Backward compatible**: When `fs.oss.credentials.provider` is not configured, the connector falls back to the existing `access_key`/`access_secret` behavior. No breaking changes.
2. **Provider chain**: Multiple providers can be configured as a chain. For example, try environment variables first, then fall back to ECS RAM role:
```hocon
fs.oss.credentials.provider = [
".EnvironmentVariableCredentialsProvider",
"OssInstanceProfileCredentialsProvider"
]
```
3. **hadoop-aliyun integration**:
- Single provider → set directly on `fs.oss.credentials.provider` (native hadoop-aliyun key)
- Multiple providers → use a custom `OssCredentialProviderChain` class that is set as the provider, with the chain list stored in a separate configuration key `fs.oss.credentials.provider.chain`
4. **Wrapper providers**: The OSS SDK's `InstanceProfileCredentialsProvider(String roleName)` and `DefaultCredentialProvider(ak, sk, token)` constructors are incompatible with hadoop-aliyun's reflection mechanism `(URI, Configuration)`. Two wrapper classes are needed:
- `OssInstanceProfileCredentialsProvider(URI, Configuration)` — reads `role_name` from Configuration and delegates to the SDK's InstanceProfileCredentialsProvider
- `OssStsCredentialsProvider(URI, Configuration)` — reads AK/SK/Token from Configuration and delegates to the SDK's DefaultCredentialProvider
5. **Sink factory change**: `access_key` and `access_secret` change from `required` to `optional` in OssFileSinkFactory (Source already has them as optional).
### Usage Scenario
1. **ECS deployment (most common)**: When SeaTunnel runs on Alibaba Cloud ECS instances, users want to avoid hardcoding AK/SK in config files. Using ECS RAM roles allows automatic, credential-free access to OSS with automatic key rotation.
2. **Temporary access**: When third-party applications need temporary OSS access, STS tokens provide time-limited credentials that automatically expire, improving security.
3. **Multi-environment deployment**: Using provider chains allows the same configuration to work across different environments (e.g., ECS in production with RAM role, local development with environment variables).
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.