OAuth client sends discovery/registration requests with no User-Agent, so WAF-fronted servers (Cloudflare) 403 the whole flow

Bắt đầu trong mcp/client/auth/utils.py tại create_oauth_metadata_request và create_client_registration_request, sau đó theo dõi cách OAuthClientProvider.async_auth_flow gửi các request của chúng. Sử dụng bản tái hiện bằng MockTransport để so sánh các header với client.get(); hoàn tất khi các request discovery và registration không còn làm mất User-Agent bắt buộc, đồng thời OAuth flow kết quả đã được xác minh bằng tests hoặc một transport cục bộ tương đương.

Summary

OAuthClientProvider builds its OAuth discovery and registration requests as bare httpx.Request objects and sends them via client.send(). httpx merges a client's default headers only in build_request() (i.e. for client.get() / .post()), so these requests go out with no User-Agent and no Accept header.

Any MCP server behind a WAF that blocks user-agent-less traffic — Cloudflare's default bot management does — answers 403 to every one of them, making OAuth impossible to complete. The resulting error is also misattributed, which makes it hard to diagnose.

Reproduction

Against a Cloudflare-fronted MCP server (observed on https://mcp.services.biorender.com/mcp), with mcp==1.26.0:

POST /mcp                                        -> 401  (expected; carries www-authenticate)
GET  /.well-known/oauth-authorization-server     -> 403  <-- blocked
GET  /.well-known/oauth-protected-resource/mcp   -> 403  <-- blocked
GET  /.well-known/oauth-protected-resource       -> 403  <-- blocked
GET  /.well-known/oauth-authorization-server     -> 403  <-- blocked
POST /register                                   -> 403  <-- note the path

Isolating the trigger with curl against that same discovery URL:

normal curl (UA + Accept present)   -> 200
curl -H 'User-Agent:' -H 'Accept:'  -> 403
curl -H 'User-Agent:'               -> 403   # UA alone is the trigger
curl -H 'Accept:'                   -> 200   # Accept is not

And confirming the header loss is structural, not server-specific:

import httpx, asyncio

seen = {}
def handler(req):
    seen[req.url.path] = dict(req.headers)
    return httpx.Response(200, json={})

async def main():
    async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as c:
        await c.get('https://x.test/normal')                       # client.get()
        await c.send(httpx.Request('GET', 'https://x.test/bare'))   # what the SDK does

asyncio.run(main())
print('client.get() :', sorted(seen['/normal']))
print('client.send():', sorted(seen['/bare']))

client.get() : ['accept', 'accept-encoding', 'connection', 'host', 'user-agent']
client.send(): ['host']

Secondary problem: the failure is misreported

The 403 lands on metadata discovery, so context.oauth_metadata stays None. create_client_registration_request then falls back to urljoin(auth_base_url, "/register") — but this server's actual registration endpoint is /oauth/register, which its discovery document advertises correctly and which works fine when called with a User-Agent.

So the error surfaced to the user is Registration failed: 403 <cloudflare html>, pointing at a registration request to a path that was never the right one, when the real failure was four requests earlier. Anyone debugging this starts at the wrong end. (This is arguably worth addressing independently: a discovery failure could be reported as a discovery failure rather than silently degrading into a guessed-path registration.)

Affected code

• mcp/client/auth/utils.py:211 — create_oauth_metadata_request -> Request("GET", url, headers={MCP_PROTOCOL_VERSION: ...})
• mcp/client/auth/utils.py:215-227 — create_client_registration_request -> Request("POST", registration_url, json=..., headers={"Content-Type": "application/json"})

Both construct Request outside the client, so neither inherits client defaults.

Suggested fix

Set a default User-Agent (e.g. mcp-python-sdk/<version>) on the requests these helpers build, or have OAuthClientProvider.async_auth_flow stamp one onto each request it yields when absent. Adding Accept: application/json to discovery would also be reasonable, though it is not what triggers the block here.

Workaround

Downstream clients can pass an httpx_client_factory that attaches a request event hook, since hooks fire for every request the client sends, including the auth flow's bare ones:

async def _stamp_user_agent(request: httpx.Request) -> None:
    if "user-agent" not in request.headers:
        request.headers["user-agent"] = "my-app/1.0"

def factory(headers=None, timeout=None, auth=None):
    client = create_mcp_http_client(headers=headers, timeout=timeout, auth=auth)
    client.event_hooks = {"request": [_stamp_user_agent], "response": []}
    return client

Environment

• mcp 1.26.0, httpx 0.28.1, httpx-sse 0.4.1, anyio 4.11.0
• Python 3.11.13, Linux