developmentseed / developmentseed/multistore

Outbound backend credential federation (OIDC identity → backend cloud STS)

Open
#56 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
19
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Summary

multistore federates identity **inbound** (callers → proxy, via `multistore-sts` + `multistore-oidc-provider`). The symmetric **outbound** half is missing: the proxy presenting *its own* identity to a **backend cloud** to obtain short-lived credentials for a private bucket.

Today a backend is either anonymous (`anonymous_access` / `skip_signature`) or configured with **long-lived static keys** in `BucketConfig.backend_options`. This issue tracks adding first-class **outbound credential federation** so deployments can serve private buckets without storing long-lived backend secrets. PRs below reference this issue.

## Motivation

- Operators increasingly want to front **customer-owned** buckets. Long-lived keys at the edge are a liability: rotation, blast radius, secret-at-rest.
- With OIDC federation the bucket config holds only a **role ARN**; the proxy assumes a short-lived, optionally prefix-scoped role at request time. Trust and blast radius are governed by the *bucket owner's* IAM policies, not the proxy's correctness.
- multistore already has the building blocks — it can act as an OIDC provider and already does inbound STS — so this simply completes the symmetry.

## Design

At bucket-resolution time, per backend:

1. Mint a short-lived OIDC assertion with the proxy's signing key (`multistore-oidc-provider`), scoped via its `aud` / `sub` claims.
2. Exchange it at the backend cloud's STS — AWS `AssumeRoleWithWebIdentity`, with Azure / GCS workload-identity analogs — for temporary credentials.
3. Inject those into `BucketConfig.backend_options` so the existing backend signs requests with them instead of going anonymous.

No long-lived secret is stored anywhere; credentials are short-lived and refreshed before expiry.

## Constraints discovered while prototyping

- **S3 needs no core change.** object_store's `AmazonS3ConfigKey::Token` accepts the `token` alias, and `token` is already in `REDACTED_OPTION_KEYS`, so temporary creds (access key + secret + session token) flow through and are redacted in logs.
- **Azure / GCS need a bearer-credential source.** Workload-identity federation yields an OAuth **bearer token**, but the Azure backend takes a shared `access_key` and GCS a `service_account_key` — neither accepts a bearer token. Parity requires accepting bearer/access-token credentials (likely an object_store contribution).
- **Caching matters and is subtle.** Short-lived creds must be cached and refreshed per backend, *independent of the caller*. Anything varied at assume-time is baked into the credential and fragments the cache, so the cache key should track the **credential identity** (role / subject), not the caller. This is a generic concern every integrator would otherwise reinvent.

## Roadmap

- [ ] **Federation exchange primitive** — new `multistore-federation` crate: a runtime-agnostic `AssumeRoleWithWebIdentity` request builder + XML response parser (caller owns HTTP), and `FederatedCredentials::apply_to(&mut BucketConfig)`. Additive, no core changes. _Branch: `feat/backend-federation`._ Validated end-to-end against real AWS STS in a downstream deployment (mint → `AssumeRoleWithWebIdentity` → signed `ListObjectsV2`, including a `sub` trust-policy condition).
- [ ] **Caching credential provider** — a `BackendCredentialProvider` trait + a caching wrapper (TTL, refresh-ahead, single-flight) so integrators get correct, non-thundering-herd refresh for free.
- [ ] **Azure / GCS bearer credential source** — accept OAuth bearer tokens for the Azure and GCS backends to bring them to parity with S3.
- [ ] **Docs / example** — wire federation into the example server / cf-workers runtimes.

## Out of scope

Application-specific **policy** — which role to use for which bucket, and how the OIDC `sub` is structured/scoped — stays in the integrator's `BucketRegistry`. multistore provides the **mechanism**, not the policy.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.