OAuth AS metadata discovery crashes on 200 OK + non-JSON response instead of falling back to OIDC

Description

When performing OAuth Authorization Server metadata discovery, if an endpoint returns 200 OK with a non-JSON body (e.g., HTML index page — which commonly happens when a reverse proxy catch-all redirect leads to a static HTML page), the SDK throws a SyntaxError from response.json() instead of treating it as a failed endpoint and continuing to the next URL in the discovery priority list.

Per RFC 8414 §3.2:

"A successful response MUST use the 200 OK HTTP status code and return a JSON object using the application/json content type"

200 OK + HTML is not a successful response by definition. The SDK must treat it as an endpoint failure.

Per MCP authorization spec:

"MCP clients MUST attempt multiple well-known endpoints when discovering authorization server metadata."

The fallback to OIDC endpoints is mandatory regardless of the failure reason on the previous endpoint.

Steps to reproduce

1. MCP server serves OAuth 2.0 Protected Resource Metadata (RFC 9728) pointing to an OAuth AS with a path component:

   { "authorization_servers": ["https://auth.example.com/oauth-path"] }
   

2. OAuth AS exposes OIDC discovery at:

   https://auth.example.com/oauth-path/.well-known/openid-configuration
   

3. Requesting the RFC 8414 endpoint returns 302 followed by 200 OK + HTML:

   GET https://auth.example.com/.well-known/oauth-authorization-server/oauth-path
   → 302 → 200 OK + HTML
   

4. MCP client initiates connection → receives 401 with resource_metadata → fetches Protected Resource Metadata → extracts AS URL → attempts RFC 8414 discovery → crash
5. OIDC endpoint https://auth.example.com/oauth-path/.well-known/openid-configuration is never attempted

Expected behavior

200 OK + non-JSON body → JSON decode fails → treat as endpoint failure → continue to next URL in discovery order → successfully discovers via:

https://auth.example.com/oauth-path/.well-known/openid-configuration

Proposed fix

let json: unknown;
try {
  json = await response.json();
} catch {
  // Body is not JSON → not a valid metadata document per RFC 8414 §3.2
  await response.body?.cancel();
  continue;
}

if (type === 'oauth') {
  return OAuthMetadataSchema.parse(json);
} else {
  return OpenIdProviderDiscoveryMetadataSchema.parse(json);
}

Additional context

RFC 8414 was designed with the assumption that the AS owns the domain and can serve /.well-known/ at the root. In practice, AS is often just a tenant on a shared domain — deployed at a subpath alongside other services behind a common reverse proxy.

The MCP spec requires MCP clients to support both RFC 8414 and OIDC Discovery, while an AS needs to provide only one of them. An AS exposing only OIDC Discovery is fully spec-compliant. The OIDC path-append variant:

/{path}/.well-known/openid-configuration

does not require the AS to control the domain root — only its own subpath. It is one of the client's priority order, and this SDK bug prevents it from ever being reached, making spec-compliant OIDC-only AS deployments inaccessible.

A concrete example of this setup: an OAuth AS hosted on a shared domain where the operator controls only paths under /oauth-path. Root-level paths including /.well-known/ are managed by a separate service outside the operator's control and cannot be reconfigured. The only viable discovery endpoint is the OIDC one available under the operator-controlled path. The SDK's inability to fall back makes the integration impossible without changes to infrastructure the operator does not own.

The issue names no source file or test. Start by locating the OAuth Authorization Server metadata discovery entry point and reproduce the RFC 8414 request that returns 200 OK with HTML. Verify that invalid JSON is treated as a failed endpoint and discovery continues to the path-based OIDC URL, with a regression test covering the fallback.