apache / apache/shenyu

[BUG] <title>concurrent-tool-calls

Open
#7,041 4 comments 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
Java
Stars
8.8k
Forks
3.1k
Avg merge
7d 1h
Merged PRs (30d)
85

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Current Behavior

## Bug Report

### Which version of ShenYu?

master (verified against `f7602e324`). The affected code has been unchanged since it was introduced in `e2cb6f3ab` (2025-07-15, #5999).

### Expected behavior

Two or more MCP `tools/call` requests issued concurrently within the same MCP session should each be proxied independently and return their own result. MCP clients routinely issue parallel tool calls, and at the transport level each call is already a separate HTTP POST carrying its own JSON-RPC `id`.

### Actual behavior

**Every concurrent tool call fails.** Not a rare race — a 100% failure rate in my tests. Failures surface as three different errors that all look like downstream/network problems:

```
{"code":-103,"message":"Service invocation exception, or no result is returned!"}
{"code":-106,"message":"Can not find url, please check your configuration!"}
"" (empty response)
Tool execution failed: ... NullPointerException: Cannot invoke "java.lang.Long.longValue()"
```

Responses can also be truncated mid-JSON:

```
{"jsonrpc":"2.0","id":"p-P1","result":{"content":[{"type":"text","text":"{\"code\
^ stream cut, 81 bytes total
```

### How to reproduce

1. Configure an `mcpServer` selector with one tool whose `requestConfig` proxies a POST endpoint that echoes its request body, e.g.

```json
{
"name": "echo_tool",
"parameters": [{ "name": "note", "type": "string", "description": "echoed back" }],
"requestConfig": "{\"requestTemplate\":{\"url\":\"/echo\",\"method\":\"POST\",\"argsToJsonBody\":true,\"headers\":[]},\"argsPosition\":{\"note\":\"body\"}}"
}
```

2. Open one MCP session:

```bash
GW=http://:9195//streamablehttp
SID=$(curl -sD- -o/dev/null -X POST "$GW" \
-H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"probe","version":"1.0"}}}' \
| grep -i '^Mcp-Session-Id:' | tr -d '\r' | awk '{print $2}')
curl -s -o/dev/null -X POST "$GW" -H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' -H "Mcp-Session-Id: $SID" \
-d '{"jsonrpc":"2.0","method":"notifications/initialized"}'
```

3. Baseline — call the tool **serially** twice with distinct `note` values. Both succeed and each response carries its own `note`.

4. Now fire three calls **concurrently on the same session**:

```bash
for n in X Y Z; do
curl -s -X POST "$GW" -H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' -H "Mcp-Session-Id: $SID" \
-d "{\"jsonrpc\":\"2.0\",\"id\":\"c-$n\",\"method\":\"tools/call\",\"params\":{\"name\":\"echo_tool\",\"arguments\":{\"note\":\"$n\"}}}" &
done; wait
```

### Results observed

| scenario | outcome |
| --- | --- |
| 2 serial calls (before concurrency) | 2/2 correct, each response matched its own `note` |
| **3 concurrent × 3 rounds** | **9/9 failed, 0 succeeded** |
| **2 concurrent** | **2/2 failed** (one `-103`, one truncated response) |
| 2 serial calls (after concurrency) | 2/2 correct — the session is not poisoned; failures are strictly concurrent-only |

### Root cause

Each concurrent tool call arrives as its own HTTP POST and therefore already has its own `ServerWebExchange`. That isolation is then discarded: the exchange is stored in a static map keyed by **session id**, so N concurrent requests collapse into one slot.

`ShenyuMcpExchangeHolder`:

```java
private static final Map EXCHANGE_MAP = new ConcurrentHashMap<>();

public static void put(final String sessionId, final ServerWebExchange exchange) {
EXCHANGE_MAP.put(sessionId, exchange); // later request overwrites the earlier one
}
```

`ShenyuStreamableHttpServerTransportProvider#configureExchangeForSession` (line 566) stores every POST's exchange under that single key, and `ShenyuToolCallback#call` (line 134) reads it back by session id:

```java
final String sessionId = extractSessionId(mcpExchange);
final ServerWebExchange originExchange = getOriginExchange(sessionId);
final ShenyuPluginChain chain = getPluginChain(originExchange);
```

Because the tool call reuses the **inbound** exchange and replays the plugin chain on it, all per-request state lives on that now-shared object and concurrent calls overwrite each other's attributes. Each observed error maps to one clobbered attribute:

| error | attribute lost | site |
| --- | --- | --- |
| `-106 Can not find url` | `HTTP_URI` (written by `URIPlugin`) | `AbstractHttpClientPlugin:67` |
| `-103 no result` | `CLIENT_RESPONSE_CONN_ATTR` | `NettyClientMessageWriter:60` |
| empty / truncated body | response written by two writers | `NettyClientMessageWriter` `response.writeWith(body)` |

### Suggested fix

Either of:

1. Key the holder by the JSON-RPC **request id** (or any per-call token) instead of the session id, and clean the entry up when the call completes. MCP explicitly allows concurrent in-flight requests per session, which is exactly what the JSON-RPC `id` is for.
2. Do not reuse the inbound exchange at all — build a fresh outbound request per tool call rather than mutating and replaying the inbound one.

Option 2 also removes the need for the blocking wait in `ShenyuToolCallback:270` (`responseFuture.get(60, SECONDS)`), which currently blocks inside a reactive pipeline.

### Notes

Since the per-tool-call timeout here is 60s while a `divide` rule with the default `retry = 3` can take `4 × timeout`, the two limits can also disagree; that is a separate, smaller concern.

### Expected Behavior

_No response_

### Steps To Reproduce

_No response_

### Environment

```markdown
ShenYu version(s):
```

### Debug logs

_No response_

### Anything else?

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ShenyuMcpExchangeHolder, then trace configureExchangeForSession in ShenyuStreamableHttpServerTransportProvider and the exchange lookup in ShenyuToolCallback. Reproduce the issue with the provided concurrent curl requests and compare serial and concurrent calls. Done means concurrent calls on one MCP session each return the correct independent result without truncated or cross-request responses.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.