apache / apache/gravitino

[SPARK] Support bearer token file authentication in Spark connector

Open
#11,181 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

## Motivation

The Spark connector currently supports `simple`, `oauth2` and `kerberos` authentication when accessing a Gravitino server.

The existing `oauth2` mode requests a token through the configured OAuth2 token endpoint, which works for client-credentials style deployments. It does not cover runtime environments where a trusted platform component has already obtained a short-lived user or workload Bearer token and refreshes it outside the Spark connector.

Common examples include:

- Kubernetes notebook runtimes where a sidecar refreshes a short-lived token.
- Enterprise platforms where a runtime credential service writes a token file for the local Spark driver.
- Workload identity systems where Spark should reuse an existing Bearer token instead of holding OAuth client credentials.

In these cases, the Spark connector only needs to read the local token and send it as:

```text
Authorization: Bearer
```

The Gravitino server still validates the token and enforces authorization. The server does not read the token file.

## Proposal

Add a Spark connector authentication mode:

```properties
spark.sql.gravitino.authType=bearer-token-file
spark.sql.gravitino.bearer.tokenFile=/path/to/token
```

Behavior:

- The token file is local to the Spark driver.
- The file may contain either the raw token or a `Bearer ` prefixed value.
- The connector reads the token file when generating authentication data for Gravitino client requests.
- The implementation uses the existing `CustomTokenProvider` extension point.
- The feature is generic and does not depend on JupyterHub, Kubernetes, a specific identity provider, or a specific token issuer.

## Why not use the existing OAuth2 mode?

The existing OAuth2 mode requires the Spark connector to call the OAuth2 token endpoint with configured credentials. In the runtime-token-file pattern, token acquisition and refresh are handled by a trusted runtime component, and the connector should not need OAuth client credentials or refresh tokens.

## Scope

This issue only proposes a Spark connector client-side authentication mode. It does not propose any server-side token issuance behavior or any deployment-specific sidecar implementation.

Contributor guide

Open the contributing guide

Research direction

Start in the Spark connector authentication configuration and the existing CustomTokenProvider extension point; trace how authType and token properties become Gravitino client authentication data. Verify raw and Bearer-prefixed token files and the resulting request authorization in the connector tests; done means the new mode works without OAuth client credentials.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
authentication, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.