MCPRemoteProxy: persist/refresh unattended upstream OAuth for always-on clients

Problem

MCPRemoteProxy can front an OAuth-only remote MCP (RFC 7591 DCR + authorization code / device code + refresh_token). For unattended clients (CI, scheduled agents, always-on hubs) that is not enough:

1. Embedded auth still requires each client to complete an interactive login against the proxy. Tokens live in the auth-server store, not as a shared upstream credential the proxy can inject on every request.
2. Header injection / bearer from Secret (headerForward.addHeadersFromSecret, MCPExternalAuthConfig bearer) covers static API keys. There is no first-class way to:
   • persist an upstream refresh_token in a Secret,
   • refresh it before expiry,
   • inject the current access token on outbound MCP calls,
     without an out-of-band CronJob/operator that the user has to build.
3. CLI CachedRefreshTokenRef exists for local thv, but the Kubernetes operator does not expose an equivalent on MCPRemoteProxy for a shared unattended identity.

The gap is: OAuth-only remotes cannot be “permanently logged in” for machine clients without either (a) every client doing browser/device login, or (b) a homegrown refresh loop writing Secrets.

Related: #3111 (bearer for remotes — shipped for static tokens, not refresh).

Proposal

Add an operator-managed “unattended upstream OAuth” mode on MCPRemoteProxy (or MCPExternalAuthConfig), for example:

• Secret refs for refresh_token + DCR client_id (and optional client_secret)
• Token endpoint / resource from OAuth discovery (already used for interactive DCR)
• Periodic refresh inside the proxy-runner (or a controller reconcile)
• Inject Authorization Bearer access tokens on outbound MCP requests
• Re-auth signal (condition + event) when refresh fails, instead of silently 401ing clients

This is distinct from per-user embedded auth. It is a single service identity for always-on hubs.

Alternatives considered

• Static bearer Secret: many OAuth MCPs do not issue long-lived PAT-style tokens.
• Clients refresh themselves and send Bearer: duplicates DCR/login in every harness.
• External CronJob writing Secrets: works, but every operator user reinvents it.

Environment

• ToolHive Kubernetes operator (MCPRemoteProxy + embeddedAuthServer)
• Remote MCP using OAuth 2.1 / DCR / refresh_token (no static API key)

Please do not ask for tenant URLs, token dumps, or client credentials in comments.

Start by tracing MCPRemoteProxy, MCPExternalAuthConfig, embeddedAuthServer, and the proxy-runner paths that handle outbound MCP authentication; compare them with CLI CachedRefreshTokenRef. Define how Secret refs, OAuth discovery, refresh scheduling, and failure conditions fit the operator design. Done means an unattended shared identity can refresh and inject access tokens, with an observable re-auth signal when refresh fails.