feast-dev / feast-dev/feast

feat: OIDC client auth should accept a refreshable token source (callable or file) — a caller that computes its own token can only pass a static string

Open
#6,667 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
3d 16h
Merged PRs (30d)
15

Description

**Is your feature request related to a problem? Please describe.**

OIDC access tokens are short-lived by design. No identity provider issues one that
outlives a long-running process, and for machine identities the client-credentials
flow issues no refresh token at all — the caller is expected to hold a credential
and request a fresh token whenever it needs one. Every provider SDK is built
around that: you keep a credential object and call it per use, and it caches and
renews underneath.

Feast's client auth is *almost* aligned with this. `get_token()` is called on
every request — `GrpcClientAuthHeaderInterceptor._append_auth_header_metadata`
calls `get_auth_token()` per RPC, and `HttpSessionManager.get_session()` refreshes
the `Authorization` header per call — and three of the four sources in
`OidcAuthClientManager.get_token()` genuinely re-read each time:

- `token_env_var` → `os.getenv(...)` on every call
- `FEAST_OIDC_TOKEN` → same
- `_read_sa_token()` → re-opens `/var/run/secrets/kubernetes.io/serviceaccount/token` on every call, which is what makes it survive kubelet's rotation of that file

The exception is `token`, an `Optional[str]` on `OidcClientAuthConfig`. It is the
only source available to a caller that obtains the token *in process* — from a
provider SDK, a credential object, a token-exchange call — and it is the one
source that cannot change. A store built with it holds that exact string for its
lifetime, so the process starts failing with 401s at expiry and has no way to hand
Feast a new value. The per-call re-read that makes the other sources work is
wasted on it.

So the shape of the gap is: Feast supports refresh only for token sources *it*
knows how to poll. A caller that knows how to mint a valid token has no way to
pass that capability in, only a snapshot of its output. The mutual-exclusivity
check in `_validate_credentials` makes this explicit — `token`, `token_env_var`
and the client-credentials group are the complete set of choices.

Two consequences we hit in practice:

1. *Long-lived clients.* Our only workaround is to discard and rebuild the entire
`FeatureStore` per call, purely so our own token acquisition re-runs. That
throws away store construction every time, and it is not viable at all for a
long-lived server process that holds a store.

2. *Federated / workload identity.* Any client-assertion flow (RFC 7523) exchanges
a rotating, projected credential for an access token, with no client secret
involved. `client_secret` therefore does not apply, and `token` cannot express
a value that changes. This is why our UI component ends up monkeypatching
`get_token()` outright — there is no configuration that expresses "call this to
get a token."

**Describe the solution you'd like**

A refreshable token source for in-process callers. In rough order of how cheap
they are:

1. **A `token_file` field**, read on each `get_token()` call. This is a
generalisation of `_read_sa_token()`, which already does exactly this for one
hardcoded path — the behaviour is proven in the codebase, it just isn't
available for any other file. It covers every projected-credential and
sidecar-written-token setup declaratively, with no code and no new concepts.

2. **An optional callable**, e.g. `token_provider: Callable[[], str]`, set
programmatically, which `get_token()` invokes when present. Since `get_token()`
is already called per request, this needs no transport changes, and it hands
the refresh problem to the party that can actually solve it — the caller, who
owns the credential. This subsumes every provider-specific case at once and
would let Feast stop growing per-flow configuration.

3. **Optionally, the client-assertion grant** in `_fetch_token_from_idp`: a
`client_assertion_file` posted as `client_assertion` with
`client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer`.
This makes the common secretless machine-identity case work with configuration
alone, rather than requiring (2).

(1) and (2) are additive and independent; either one on its own resolves the
blocker.

**Describe alternatives you've considered**

- *Request a longer-lived token from the provider.* Where a provider allows the
access-token lifetime to be raised at all, the ceiling is still far short of a
process that runs for days, and raising it widens the exposure of a leaked
token. This trades a security property for a library limitation.
- *Rebuild the store per call.* What we do. Correct, but it discards store
construction on every use and does not work for a long-lived process.
- *Write the token into an environment variable before each call.* Would work,
since `token_env_var` is re-read — but it means mutating global process state
from library code, and coordinating that mutation with every caller and thread.
- *Point `token_file` at a file we write ourselves.* Not currently possible; only
the hardcoded Kubernetes SA path is read. This is what suggestion (1) unblocks.
- *Subclass `OidcAuthClientManager`.* `AuthenticationClientManagerFactory`
selects the manager from `auth_config.type`, a
`Literal["oidc", "kubernetes", "no_auth"]`, so a custom manager cannot be
selected.
- *Monkeypatch `get_token()`.* What we currently do for our UI component, via a
`sitecustomize.py` mounted on `PYTHONPATH` in the container. It works and is
plainly not something we want to maintain.

**Additional context**

Version 0.64.0; the three credential shapes are identical on 0.65.0, the latest
release. We run against Azure Entra with OIDC auth on the registry, online and
offline servers. Callers are Kubernetes pods using workload identity plus
developer machines, so nothing in the deployment holds a static token — but the
constraint is not Azure-specific. The same problem applies to any caller using a
provider SDK, and the Kubernetes SA token would have it too if
`_read_sa_token()` did not already re-read the file.

Happy to contribute (1) and (2) — both are small, and `_validate_credentials` is
an obvious place to slot additional sources into the existing mutual-exclusivity
check.

Contributor guide

Open the contributing guide

Research direction

Start with OidcAuthClientManager.get_token(), _validate_credentials(), and _fetch_token_from_idp(), then trace the calls from GrpcClientAuthHeaderInterceptor and HttpSessionManager. Clarify which proposed refreshable source is in scope, preserve mutual exclusivity, and verify that the selected source is re-read or invoked per request without regressing existing authentication behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.