anthropics / anthropics/claude-agent-sdk-python

ToolSearch from deferred tool loading is not blocked by disallowed_tools, and costs ~10x on single-tool sessions

Đang mở
#1,147 2 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
8.1k
Fork
1.3k
Merge trung bình
2 ngày 31 phút
Pull request đã merge (30 ngày)
1

Mô tả

### Summary

When a session exposes exactly one in-process MCP tool and lists the built-in tools in `disallowed_tools`, the CLI still loads a built-in `ToolSearch` meta-tool through deferred tool loading. The model then calls `ToolSearch` to discover the single MCP tool that was already named in `allowed_tools`. This costs an extra API round trip and about **10x** the total cost of the identical task run with `tools=[]`.

`ToolSearch` is not in the public built-in tool list, so a user writing a denylist has no way to know they should name it.

### Environment

- `claude-agent-sdk` 0.2.128 (latest on PyPI at time of writing)
- bundled Claude Code CLI 2.1.220
- Python 3.12.3, Linux x86_64
- model: `claude-sonnet-5`

### Reproduction

```python
import anyio
from claude_agent_sdk import (ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server,
query, tool)

BUILTINS = ["Bash", "Read", "Write", "Edit", "Glob", "Grep", "WebFetch", "WebSearch",
"NotebookEdit", "Task", "TodoWrite", "BashOutput", "KillShell", "SlashCommand"]

@tool("get_secret_number", "Returns a secret number for demonstration.", {"seed": str})
async def get_secret_number(args: dict) -> dict:
return {"content": [{"type": "text", "text": f"secret={len(args['seed'])}"}]}

server = create_sdk_mcp_server(name="demo", version="1.0.0", tools=[get_secret_number])
FULL = "mcp__demo__get_secret_number"

async def run(zero_base: bool) -> None:
kwargs = dict(
model="claude-sonnet-5",
system_prompt="Answer in one sentence. Use the demo tool when asked.",
mcp_servers={"demo": server},
allowed_tools=[FULL],
disallowed_tools=BUILTINS,
setting_sources=[],
permission_mode="bypassPermissions",
max_turns=5,
)
if zero_base:
kwargs["tools"] = [] # the only difference between the variants
async for msg in query(prompt="Call the demo tool with seed 'demo', then summarise.",
options=ClaudeAgentOptions(**kwargs)):
for block in getattr(msg, "content", []) or []:
if getattr(block, "type", None) == "tool_use":
print("tool_use:", block.name)
if isinstance(msg, ResultMessage):
print("turns:", msg.num_turns, "cost_usd:", msg.total_cost_usd, msg.usage)

anyio.run(run, False) # variant A - denylist only
anyio.run(run, True) # variant B - zero-base allowlist
```

### Observed

**Variant A** (`disallowed_tools` only):

```
tool_use: ToolSearch <- not requested, not in the public tool list
tool_use: mcp__demo__get_secret_number
turns: 3 cost_usd: 0.0683255 cache_creation_input_tokens: 14774
elapsed: 7.79 s
```

**Variant B** (identical, plus `tools=[]`):

```
tool_use: mcp__demo__get_secret_number
turns: 2 cost_usd: 0.00712225
usage: input_tokens=4, cache_creation_input_tokens=1235,
cache_read_input_tokens=1140, output_tokens=100
elapsed: 4.82 s
```

Same task, same model, same tool: **0.0683 USD vs 0.0071 USD**, a factor of 9.6, almost entirely cache-write tokens spent on loading a tool-discovery mechanism for a session that had one tool.

### Expected

Any one of these would resolve it:

1. `ToolSearch` is suppressible by name through `disallowed_tools`, like other built-ins.
2. Deferred tool loading does not engage when the session's total tool count is trivially small — discovering one tool cannot pay for itself.
3. The documentation states plainly that `disallowed_tools` is a denylist and is not sufficient for isolation, and that `tools=[]` is the way to restrict what is available.

### Why this matters beyond cost

Guides and examples commonly show `disallowed_tools` as the way to constrain a session. It is a denylist, so any built-in not named — including one absent from the published tool list — stays reachable. The type stub does say the right thing (`types.py:1770-1771`: *"To restrict which tools are available at all, use `tools`"*), but that line lives in the stub, and the option named `allowed_tools` reads as though it were the restriction. For anyone building a constrained agent, the difference between "these run without prompting" and "these are the only ones that exist" is the whole design.

The related default is worth a docs line too: `setting_sources=None` loads every filesystem source, including the operator's `~/.claude`. For a headless service that must be deterministic, `setting_sources=[]` has to be set explicitly. (#794 and #802 fixed the case where the empty list was ignored; this is about the default itself.)

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Hướng nghiên cứu

Read types.py:1770-1771, then run the reported two query variants and compare ToolSearch use, turns, and cost. Trace where deferred tool loading and disallowed_tools are handled; done means an agreed resolution prevents the unintended discovery path or clearly documents that tools=[] is required for restriction, with the reproduction covering the result.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
api, tooling
Loại issue
Lỗi
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
52/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.