apache / apache/apisix

feat(ai-proxy): support GCP Application Default Credentials / Workload Identity (metadata server) for Vertex AI auth

Open
#13,732 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Lua
Stars
17.1k
Forks
2.9k
Avg merge
3d 16h
Merged PRs (30d)
63

Description

### Description

The `ai-proxy` / `ai-proxy-multi` GCP (Vertex AI) auth currently supports **only** a static service-account JSON key. When APISIX runs on GKE with [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) (or any environment where Application Default Credentials are available), there is no way to authenticate to Vertex AI **without** mounting a long-lived SA key.

This forces users to create, store, and rotate a static SA key purely for APISIX — a downgrade from the keyless posture GKE Workload Identity otherwise provides across the cluster.

### Current behavior

`fetch_gcp_access_token` in `apisix/plugins/ai-transport/auth.lua` reads the key from `gcp.service_account_json` or the `GCP_SERVICE_ACCOUNT` env var, then hands it to `apisix/utils/google-cloud-oauth.lua`, which mints a token via the JWT-bearer grant:

```lua
-- apisix/utils/google-cloud-oauth.lua
function _M.refresh_access_token(self)
-- POST self.token_uri with
-- grant_type = "urn:ietf:params:oauth:grant-type:jwt-bearer",
-- assertion = self:generate_jwt_token() -- jwt:sign(self.private_key, {iss=self.client_email})
```

If no key is provided, `auth_conf` is `{}`, `client_email`/`private_key` are nil, and token generation fails. There is no metadata-server / ADC fallback.

### Proposed enhancement

When no `service_account_json` / `GCP_SERVICE_ACCOUNT` is configured (empty auth conf), fall back to fetching an access token from the **GCE/GKE metadata server**, which is how Application Default Credentials / Workload Identity work:

```
GET http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token
Header: Metadata-Flavor: Google
-> { "access_token": "...", "expires_in": 3599, "token_type": "Bearer" }
```

This is a small, self-contained addition to `google-cloud-oauth.lua` (a code path that, given empty config, queries the metadata endpoint instead of signing a JWT). The existing token-caching logic (`max_ttl`, `expire_early_secs`) can be reused unchanged, since the metadata response already carries `expires_in`.

Suggested config (opt-in, or auto when key absent):

```yaml
ai-proxy:
provider: vertex-ai
auth:
gcp:
use_metadata_server: true # or: infer when service_account_json/env unset
provider_conf:
project_id: my-project
region: us-east5
options:
model: gemini-2.5-flash
```

### Use case

- APISIX on GKE with Workload Identity: bind the gateway's KSA to a GCP SA (`roles/aiplatform.user`) and let ai-proxy obtain tokens keylessly — no static SA key to mount, store, or rotate.
- Aligns APISIX with how other Google-SDK-based gateways already authenticate to Vertex AI.

### Alternatives considered

- Static SA key (current) — works, but reintroduces a long-lived credential.
- External sidecar that fronts Vertex with ADC — extra moving part; defeats using ai-proxy directly.

### Environment

- APISIX gateway 3.17.x (ai-proxy plugins present in-image).
- Deployment: GKE with Workload Identity enabled.

Contributor guide

Open the contributing guide

Research direction

Start in apisix/plugins/ai-transport/auth.lua and apisix/utils/google-cloud-oauth.lua, tracing fetch_gcp_access_token and refresh_access_token. Compare the existing service-account JWT flow with the documented metadata request, then verify that the chosen empty-config behavior preserves max_ttl and expire_early_secs caching and handles the documented token fields. Done means Vertex AI authentication works with Workload Identity without a static key while the existing key path remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, kubernetes, lua
Domain
ai, authentication, cloud
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.