langgenius / langgenius/dify

Cannot authorize streamable-http MCP servers when URL path's last segment isn't mcp/sse - SSE-first fallback never triggers (httpx.ReadTimeout not caught)

Open Beginner friendly
#39,301 1 comment 1 reaction 0 assignees View on GitHub
1.15.0 project#dify
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

**Title:** Cannot authorize streamable-http MCP servers when URL path's last segment isn't `mcp`/`sse` — SSE-first fallback never triggers (httpx.ReadTimeout not caught)

---

**Dify version:** 1.15.0

**Cloud or Self Hosted:** Self Hosted (Docker)

### Steps to reproduce

1. Run Dify 1.15.0 via Docker (`langgenius/dify-api:1.15.0`).
2. Go to **Tools → MCP** and add an MCP server whose endpoint is a **streamable-http** server and whose URL path does **not** end with `/mcp` or `/sse`.
Example: a PandaWiki MCP share endpoint:
```
http://:8089/share/v1/mcp/
```
Here the last path segment is the `share_id`, not `mcp`.
3. Confirm the server is a healthy streamable-http MCP server. A direct `POST .../initialize` returns `200` + `Mcp-Session-Id` and a valid JSON-RPC initialize response:
```json
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2024-11-05","capabilities":{"tools":{"listChanged":true}},"serverInfo":{"name":"pandawiki-mcp","version":"1.0.0"}}}
```
A `GET` (SSE-style) to the same URL hangs and returns nothing.
4. Click **Authorize** on the provider.

### ✔️ Expected Behavior

Dify detects that the server speaks streamable-http, performs the `initialize` handshake, fetches the tool list, and marks the provider as authorized.

### ❌ Actual Behavior

Clicking **Authorize** appears to do nothing in the UI.

- nginx access log:
```
POST /console/api/workspaces/current/tool-provider/mcp/auth -> 499
```
(`499` = client closed the connection before the backend responded.)
- api log:
```
core/mcp/client/sse_client.py:301 Error connecting to SSE endpoint
httpcore.ReadTimeout: timed out
tool_providers.py:1120 Failed to fetch MCP tools after creation
```
- The provider stays `authed=false` in `tool_mcp_providers`. The backend blocks up to `sse_read_timeout` (default **300s**); the browser gives up first → `499` → no feedback in the UI.

### Root cause

In `api/core/mcp/mcp_client.py`, `MCPClient._initialize` picks the transport by the **last segment of the URL path**:

```python
method_name = path.rstrip("/").split("/")[-1] if path else ""
if method_name in {"mcp", "sse"}:
client_factory = connection_methods[method_name]
self.connect_server(client_factory, method_name)
else:
try:
self.connect_server(sse_client, "sse") # tries SSE first
except (MCPConnectionError, ValueError): # cannot catch httpx.ReadTimeout
self.connect_server(streamablehttp_client, "mcp")
```

Two problems combine:

1. The `else` branch tries **SSE first**. A streamable-http server does not respond to the SSE GET, so the connect hangs until `sse_read_timeout` and raises `httpx.ReadTimeout`.
2. `except (MCPConnectionError, ValueError)` does **not** catch `httpx.ReadTimeout` (nor `httpx.ConnectError`), so the fallback to `streamablehttp_client` **never triggers**. The timeout propagates and the auth call fails after a long block.

Also note the method docstring says *"Initialize the client with fallback to SSE if streamable connection fails"* — i.e. streamable-http is supposed to be the default — but the `else` branch actually tries SSE first, the opposite of the documented intent.

### Impact

Any streamable-http MCP server whose URL path's last segment is not exactly `mcp` or `sse` (URLs ending with an id / token / share_id, or carrying a query string) **cannot be authorized** in Dify 1.15.0. This is a broad class of servers, not specific to PandaWiki.

**Related:** #24297 reported the same 300s symptom in 1.7.2 (attributed to the `sse_client` thread pool) and was closed as `cant-reproduce`. In 1.15.0 the 300s has a different, clearly identifiable root cause: the protocol-detection heuristic above + the `except (MCPConnectionError, ValueError)` that cannot catch `httpx.ReadTimeout`, so the SSE -> streamable fallback never runs.

### Suggested fix

In the `else` branch, try streamable-http first and fall back to SSE on any connection failure, while letting `MCPAuthError` (HTTP 401) propagate so it isn't retried as a different transport:

```python
else:
try:
self.connect_server(streamablehttp_client, "mcp")
except MCPAuthError:
raise
except Exception:
logger.debug("MCP connection failed with 'mcp', falling back to 'sse' method.")
self.connect_server(sse_client, "sse")
```

Notes:
- `MCPAuthError` is a subclass of `MCPConnectionError`, so it must be caught and re-raised separately before the generic `except Exception` (otherwise a 401 would be silently retried over SSE).
- Verified locally: with this change, the PandaWiki server authorizes in a couple of seconds and `list_tools` returns correctly.

### Environment

- Dify 1.15.0, Self Hosted (Docker).
- `SSRF_PROXY_HTTP_URL` / `SSRF_PROXY_HTTPS_URL` are unset, so the MCP client connects directly (not via the squid SSRF proxy); the issue is unrelated to the `deny to_private_networks` squid rule.
- MCP server: PandaWiki, streamable-http, public share endpoint (no auth).

Contributor guide

Open the contributing guide

Research direction

Start in api/core/mcp/mcp_client.py at MCPClient._initialize and reproduce authorization against a streamable-http endpoint whose final path segment is not mcp or sse. Verify that streamable-http initialization succeeds without waiting for SSE, SSE fallback still works when needed, and MCPAuthError is not retried as another transport.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.