Federated credential cache is per-isolate; consider cross-instance caching (KV/DO) + single-flight

Context

Federated backend auth (#147) caches the temporary credentials the proxy gets from AssumeRoleWithWebIdentity so it doesn't re-mint a JWT + re-call AWS STS on every request. Today that cache is in-memory and per-isolate:

• multistore's CredentialCache is a HashMap<role_arn, creds> behind Arc<Mutex>, and we hold the OidcCredentialProvider in a OnceLock so it survives across requests within one Worker isolate (relies on developmentseed/multistore#69 making the provider Clone with an Arc-shared cache).

It does not use the Cloudflare Cache API, KV, or Durable Objects.

The gap

The cache does not span Worker instances:

• Each isolate mints + exchanges on its first request per role; different isolates (other colos, parallel isolates, or a recycled isolate) each do their own first call.
• The cache is lost when an isolate is evicted.
• There is also no single-flight: concurrent cold requests for the same role each do their own STS exchange.

This is fine for busy connections (isolates are long-lived and reused, creds last up to the role session duration), but wastes STS calls / adds first-request latency for cold or low-traffic connections, and offers no headroom against STS volume/rate limits at scale.

Options for cross-instance caching (if/when needed)

Security caveat (all options): this persists short-lived credentials outside the isolate. They're short-TTL and KV/DO are encrypted at rest, but it widens the blast radius — a deliberate decision, not a default.

multistore constraint: CredentialCache is in-memory only and not pluggable (no cache-backend trait). A KV/DO cache means either caching at the proxy layer (wrap the exchange, bypassing multistore's cache) or adding a pluggable cache trait upstream in multistore.

Recommendation

Keep the per-isolate in-memory cache as the default — it's cheap, correct, and captures most of the benefit. Treat cross-instance caching as measure-first: once federation is live, if STS call volume / cold-start latency shows up as a real problem, add KV (tight TTL + expiry margin) or a Durable Object (if we also want single-flight). Don't build it speculatively given the security tradeoff.

Refs

• proxy per-isolate cache: PR #147
• upstream Clone/Arc-shared cache: developmentseed/multistore#69

Start by reading proxy PR #147 and the multistore CredentialCache and OidcCredentialProvider references described here. Measure STS call volume and cold-start latency once federation is live, then use those results to determine whether cross-instance caching is warranted. Done means a deliberate caching decision is documented, rather than speculative implementation.