developmentseed / developmentseed/multistore
Outbound backend credential federation (OIDC identity → backend cloud STS)
- Dominant language
- Rust
- Stars
- 19
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
multistore federates identity **inbound** (callers → proxy, via `multistore-sts` + `multistore-oidc-provider`). The symmetric **outbound** half is missing: the proxy presenting *its own* identity to a **backend cloud** to obtain short-lived credentials for a private bucket.
Today a backend is either anonymous (`anonymous_access` / `skip_signature`) or configured with **long-lived static keys** in `BucketConfig.backend_options`. This issue tracks adding first-class **outbound credential federation** so deployments can serve private buckets without storing long-lived backend secrets. PRs below reference this issue.
## Motivation
- Operators increasingly want to front **customer-owned** buckets. Long-lived keys at the edge are a liability: rotation, blast radius, secret-at-rest.
- With OIDC federation the bucket config holds only a **role ARN**; the proxy assumes a short-lived, optionally prefix-scoped role at request time. Trust and blast radius are governed by the *bucket owner's* IAM policies, not the proxy's correctness.
- multistore already has the building blocks — it can act as an OIDC provider and already does inbound STS — so this simply completes the symmetry.
## Design
At bucket-resolution time, per backend:
1. Mint a short-lived OIDC assertion with the proxy's signing key (`multistore-oidc-provider`), scoped via its `aud` / `sub` claims.
2. Exchange it at the backend cloud's STS — AWS `AssumeRoleWithWebIdentity`, with Azure / GCS workload-identity analogs — for temporary credentials.
3. Inject those into `BucketConfig.backend_options` so the existing backend signs requests with them instead of going anonymous.
No long-lived secret is stored anywhere; credentials are short-lived and refreshed before expiry.
## Constraints discovered while prototyping
- **S3 needs no core change.** object_store's `AmazonS3ConfigKey::Token` accepts the `token` alias, and `token` is already in `REDACTED_OPTION_KEYS`, so temporary creds (access key + secret + session token) flow through and are redacted in logs.
- **Azure / GCS need a bearer-credential source.** Workload-identity federation yields an OAuth **bearer token**, but the Azure backend takes a shared `access_key` and GCS a `service_account_key` — neither accepts a bearer token. Parity requires accepting bearer/access-token credentials (likely an object_store contribution).
- **Caching matters and is subtle.** Short-lived creds must be cached and refreshed per backend, *independent of the caller*. Anything varied at assume-time is baked into the credential and fragments the cache, so the cache key should track the **credential identity** (role / subject), not the caller. This is a generic concern every integrator would otherwise reinvent.
## Roadmap
- [ ] **Federation exchange primitive** — new `multistore-federation` crate: a runtime-agnostic `AssumeRoleWithWebIdentity` request builder + XML response parser (caller owns HTTP), and `FederatedCredentials::apply_to(&mut BucketConfig)`. Additive, no core changes. _Branch: `feat/backend-federation`._ Validated end-to-end against real AWS STS in a downstream deployment (mint → `AssumeRoleWithWebIdentity` → signed `ListObjectsV2`, including a `sub` trust-policy condition).
- [ ] **Caching credential provider** — a `BackendCredentialProvider` trait + a caching wrapper (TTL, refresh-ahead, single-flight) so integrators get correct, non-thundering-herd refresh for free.
- [ ] **Azure / GCS bearer credential source** — accept OAuth bearer tokens for the Azure and GCS backends to bring them to parity with S3.
- [ ] **Docs / example** — wire federation into the example server / cf-workers runtimes.
## Out of scope
Application-specific **policy** — which role to use for which bucket, and how the OIDC `sub` is structured/scoped — stays in the integrator's `BucketRegistry`. multistore provides the **mechanism**, not the policy.
Contributor guide
Assessment
This issue has not been assessed yet.