--read-only reports a throttled tool lookup as "not available in read-only mode", permanently, for the whole session
- Dominant language
- Python
- Stars
- 352
- Forks
- 63
- Avg merge
- 12h 9m
- Merged PRs (30d)
- 19
Description
### Describe the bug
With `--read-only`, a **failed tool lookup** is reported to the caller as `Tool 'X' is not available in read-only mode.` When the upstream MCP endpoint starts returning HTTP 429, every subsequent `tools/call` in the session is denied with that message, on the same long-lived proxy session, and does not recover.
The message is actively misleading: the tools in question are annotated `readOnlyHint=true` and had been served successfully for minutes beforehand. An agent reading "not available in read-only mode" concludes the toolset has been withdrawn and stops retrying — which is what happened to us. It abandoned a run that a 30-second back-off would have completed.
In `mcp_proxy_for_aws/middleware/tool_filter.py`, `ToolFilteringMiddleware.on_call_tool`:
```python
tool = await context.fastmcp_context.fastmcp.get_tool(context.message.name)
if not tool or not getattr(tool.annotations, 'readOnlyHint', False):
raise ToolError(
f'Tool {context.message.name!r} is not available in read-only mode.'
)
```
`not tool` — the lookup did not return a tool — and "the tool is not read-only" are two different conditions with two different remedies, and they raise the same error. On a proxy server the lookup is resolved against upstream, so it is exactly the thing an upstream 429 breaks.
### Expected Behavior
A lookup that fails for a transport or throttling reason should surface as that failure, so the caller can back off and retry. Only an actual `readOnlyHint != True` on a tool that *was* resolved should produce "not available in read-only mode".
### Current Behavior
Timeline from one session, single proxy connection reused throughout (no reconnect, no re-launch — `Reusing session connection` on every call):
```
10:13:52 – 10:14:54 67 failures, all HTTP 429 from the upstream endpoint
26 of them: Tool call 'read_k8s_resource' failed: Rate limited by upstream API, please retry later.
41 of them: Tool call 'read_k8s_resource' failed: Error calling tool 'read_k8s_resource': .
10:15:58 onward 9 failures, one per distinct tool name, all:
Tool call 'X' failed: Tool 'X' is not available in read-only mode.
```
The underlying 429, from the proxy's own stderr:
```
HTTPStatusError: Client error '429 ' for url 'https://.us-east-1.api.aws/mcp'
.../site-packages/mcp/client/streamable_http.py:358 in _handle_post_request
.../site-packages/httpx/_models.py:829 in raise_for_status
```
Every one of the 9 tools denied as non-read-only is a read tool that had returned data earlier in the same session. Once the denials start, no tool recovers for the remainder of the session.
**Secondary issue in the same log, same middleware:** those 41 failures rendered as `Error calling tool 'read_k8s_resource': .` — the 429 was swallowed and the message left empty. The `HTTPStatusError` is visible in the proxy's stderr but the caller receives nothing to act on, so from the client side two thirds of the throttling is indistinguishable from an unexplained tool error. Propagating the status would make the back-off decision obvious.
### Reproduction Steps
1. Run the proxy with `--read-only` against an AWS-hosted MCP endpoint whose tools are annotated `readOnlyHint=true`.
2. Confirm the read tools work.
3. Drive enough concurrent read calls to get HTTP 429s from the endpoint (~140 calls over 4 minutes, batches of 10, was enough for us).
4. After the 429 burst, call any read tool again. It is refused with `not available in read-only mode` and stays refused for the life of the session.
Without `--read-only`, the same throttling surfaces as retryable tool errors and the session recovers.
### Possible Solution
Separate the two conditions in `on_call_tool` — let a lookup failure propagate as an upstream/transport error (or raise a distinct error naming the cause) and reserve the read-only message for a tool that was resolved and is genuinely not annotated read-only. Something like:
```python
try:
tool = await context.fastmcp_context.fastmcp.get_tool(context.message.name)
except Exception:
raise # a throttled or failed lookup is not a read-only denial
if tool is None:
raise ToolError(f'Tool {context.message.name!r} could not be resolved upstream.')
if not getattr(tool.annotations, 'readOnlyHint', False):
raise ToolError(f'Tool {context.message.name!r} is not available in read-only mode.')
```
Caching the resolved annotations after a successful `on_list_tools` would also avoid re-resolving a known-read-only tool upstream on every single call, which is what puts the lookup in the throttling blast radius to begin with.
### Additional Information/Context
`mcp-proxy-for-aws` 1.6.4 (current release), `uvx mcp-proxy-for-aws@1.6.4 https://.us-east-1.api.aws/mcp --service --region us-east-1 --read-only`, Python 3.12, running unattended in CI. Related but not the same: #159 covers tools being filtered out at list time when annotations are absent; this is enforcement at call time when the lookup itself fails.
Contributor guide
Research direction
Start in mcp_proxy_for_aws/middleware/tool_filter.py at ToolFilteringMiddleware.on_call_tool and trace how get_tool failures and resolved tool annotations become caller-facing errors. Reproduce the --read-only throttling scenario or run the relevant middleware tests; done means upstream lookup failures remain retryable and only a resolved non-read-only tool receives the read-only denial.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100