apache / apache/gravitino

[Subtask] Support GCS credential vending for Hive catalog in Spark connector

Open
#12,629 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Describe the subtask

When a Hive catalog is configured with `credential-providers=gcs-token` and `gcs-service-account-file`, the Gravitino server correctly vends a `GCSTokenCredential` (OAuth2 access token). However, the Spark connector's Hive path does not consume it.

In `spark-connector/spark-common/.../hive/GravitinoHiveCatalog.java`, `applyS3Credential()` only handles `S3SecretKeyCredential`, `OSSSecretKeyCredential`, and `AzureAccountKeyCredential`. Any other credential type — including `GCSTokenCredential` — falls into the `else` branch and is silently dropped with a warning:

```
Received unrecognized credential type '...' for Hive catalog, skipping
```

As a result, Spark jobs reading GCS-backed Hive tables fail with authentication errors even though vending is enabled.

Unlike S3/OSS/Azure (simple key/secret pairs injected as Hadoop config), GCS requires setting `fs.gs.auth.access.token.provider.impl` to a class implementing `com.google.cloud.hadoop.util.AccessTokenProvider`, so a small provider implementation is needed to bridge the pre-vended token into the GCS connector.

### Parent issue

#5063

Contributor guide

Open the contributing guide

Research direction

Start in spark-connector/spark-common/.../hive/GravitinoHiveCatalog.java at applyS3Credential(), then review the GCS AccessTokenProvider contract and the existing credential branches. Done means a vended GCSTokenCredential is consumed for Hive catalogs and Spark's GCS connector receives the provider configuration instead of the unrecognized-credential warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, java
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.