[Feature] Add Alibaba Cloud OSS C++ SDK stack
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 531
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
Add native Alibaba Cloud OSS support as a first-class storage backend in Doris, using the official Alibaba Cloud OSS C++ SDK instead of the existing S3-compatible (AWS SDK) path.
### Use case
Doris users running on Alibaba Cloud ECS face three problems with the current S3-compatible OSS path:
1. **No credential-free access** — The AWS SDK cannot talk to the ECS instance metadata service (100.100.100.200). Users must manage static AK/SK keys even when an ECS RAM role is attached to the instance.
2. **No STS AssumeRole** — Cross-account access and least-privilege patterns require `acs:ram::account:role/name` ARNs which are Alibaba-specific and not reachable via the AWS STS endpoint.
3. **No RRSA (OIDC)** — Kubernetes workloads on ACK use RRSA (RAM Roles for Service Accounts) for pod-level identity. The AWS SDK has no awareness of Alibaba's OIDC token file mechanism.
**Target users:**
- Doris cloud-mode deployments on Alibaba Cloud ECS
- Enterprises using RAM roles for zero-credential rotation policies
- ACK (Alibaba Container Service for Kubernetes) workloads using RRSA
- Cross-account OSS access patterns using STS AssumeRole
**Concrete scenarios:**
- Storage vaults backed by OSS with ECS instance profile (no AK/SK in config)
- Cloud recycler cleaning up OSS garbage with RAM role credentials
- Catalog/external table reads from OSS with STS-assumed least-privilege role
### Related issues
No existing issue. This feature was discussed informally and a draft PR (#61329) has been submitted. This issue formalizes the feature request and tracks the work across 4 incremental PRs:
- PR-1: #61329 FE `fe-filesystem-oss` SPI — ECS RAM role, AssumeRole, RRSA, `oss.credentials_provider` type-based dispatch
- PR-2: #65794 Thirdparty C++ SDK stack — apr, tea-cpp, credentials-cpp, openapi-v2, sts-20150401, alibabacloud-oss-cpp-sdk
- PR-3: BE native OSS filesystem — `OSSFileSystem`, `OSSFileReader`, `OSSFileWriter`, credential providers, `enable_oss_native_sdk` config flag
- PR-4: Cloud recycler OSS accessor — `OSSAccessor`, `OssObjClient`, reuses shared `delete_objects_recursively_()` helper
**Design principle:** All changes are additive and gated behind `enable_oss_native_sdk = false` (default). Existing S3-compatible OSS path is fully preserved when the flag is off.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start with draft PR #61329 and the third-party SDK work in PR #65794, then trace the named fe-filesystem-oss SPI and the planned OSSFileSystem, OSSFileReader, OSSFileWriter, credential providers, and OSSAccessor components. Done means the four incremental PRs provide native Alibaba OSS support behind enable_oss_native_sdk=false while preserving the existing S3-compatible path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100