apache / apache/doris

[Feature] Support Google Authentication for Iceberg REST Catalog

Open
#62,987 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Description

Add support for Google Authentication Manager (`GoogleAuthManager`) when connecting Doris to a Google Lakehouse Iceberg REST Catalog. Currently, the Iceberg REST catalog only supports `none` and `oauth2` security types. This feature adds a `google` security type that leverages Iceberg's built-in `GoogleAuthManager` (from the `iceberg-gcp` module) to authenticate using Application Default Credentials (ADC).

New configuration properties:
- `iceberg.rest.security.type` - Add `google` as a new option alongside existing `none` and `oauth2`
- `iceberg.rest.io-impl` - Set the FileIO implementation (e.g., `org.apache.iceberg.gcp.gcs.GCSFileIO`)
- `iceberg.rest.google.user-project` - Set the Google project to be billed for API requests. This is required!
- `iceberg.gcs.oauth2.token` - Provide an OAuth2 token for GCS storage access

Example usage:
```sql
CREATE CATALOG `lakehouse` PROPERTIES (
"type" = "iceberg",
"iceberg.catalog.type" = "rest",
"iceberg.rest.uri" = "https://my-lakehouse.example.com",
"iceberg.rest.security.type" = "google",
"iceberg.rest.io-impl" = "org.apache.iceberg.gcp.gcs.GCSFileIO",
"iceberg.rest.google.user-project" = "my-billing-project",
"warehouse" = "gs://my-bucket/warehouse/"
);
```

### Use case

Users running Google Cloud Dataproc or Google Lakehouse with Iceberg REST catalogs need a way to authenticate using Google's native authentication mechanism. With this support, Doris can seamlessly connect to Google-managed Iceberg catalogs using VM credentials (Application Default Credentials) without manually managing OAuth2 tokens or credentials. Ref: https://docs.cloud.google.com/lakehouse/docs/lakehouse-iceberg-rest-catalog#configure-catalog

### Related issues

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.