CommandCodeAI / CommandCodeAI/command-code
Tool calls silently fail on strict OpenAI-wire BYOK providers: web_search/MCP tools announced in text or empty completion instead of tool_use — undeclared-tool progressive disclosure not tolerated by all servers
@ahmadbilaldev đang làm issue này rồi.
Từ ngày 17/9/2026.
- Ngôn ngữ chính
- Không có dữ liệu ngôn ngữ
- Star
- 4k
- Fork
- 350
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
Summary
Command Code uses progressive tool disclosure: most tool schemas (web_search, web_fetch, all mcp__* tools) are not declared in the request's tools array. Their schemas are delivered lazily as the text result of a search_tools call, and the model is expected to emit a tool_calls block for a tool that is not present in the OpenAI tools parameter of that request.
Some OpenAI-wire serving stacks tolerate this (emitting tool_calls for undeclared, in-prompt-described tools); others — in our case NeuralWatt's GLM serving (glm-5.3-flash, glm-5.2) and Openrouter — strictly constrain emission to declared tools. On those servers the model either writes a text reply announcing the call ("Now I'll run the web search…") with finish_reason: "stop" and zero tool_calls deltas, or returns a completely empty completion. The harness's continuation recovery then burns turns retrying, and the run ends with the announcement as finalText without ever executing the tool.
The failure was reproduced client-free: replaying the exact captured request body directly to the provider with curl/a raw script produces the same non-emission 100% of the time (0/10), and adding the tool to the request's tools array fixes it 100% of the time (10/10) with everything else identical.
Expected Behavior
The model should emit a tool_calls block for web_search (and MCP tools) after their schemas have been loaded via search_tools, the harness should execute them, and the run should complete normally — as it does on providers that tolerate undeclared-tool calling.
Actual Behavior
- Assistant streams reasoning planning the tool call, then streams text announcing it ("Now I'll run the web search for…") and the turn ends with
finish_reason: "stop"— notool_callsdeltas in the stream. - Sometimes the completion is entirely empty (no content, no reasoning) with
finish_reason: "stop". - The harness fires
continuation_recovery(kind: "empty"/kind: "intent") and retries; failures are sticky within a session and the run often ends atmax_turnswith the announcement asfinalText. - Only tools whose schemas are declared in
tools(e.g.search_tools,shell_command,read_file) work reliably, matching the "simple tools work, web/MCP tools don't" symptom. - A/B replay matrix on the captured failing payload (5 repeats per variant, both
glm-5.3-flashandglm-5.2):- Captured payload as-is → 0/10
tool_calls(text-announce or empty) - Same payload with continuation nudge removed → 0/10 (all empty)
- Same payload +
web_searchdeclared intools→ 10/10tool_calls - Minimal payload (same system+user, only
web_searchtool) → 10/10tool_calls
- Captured payload as-is → 0/10
Context size is ruled out: the minimal-payload variant succeeds even though the failing payload carries the full ~96 KB harness context; the deciding factor is solely whether the tool is declared in tools.
Steps to reproduce the issue
- Attach any MCP server and use a BYOK provider on the OpenAI-completions wire whose serving enforces declared-tool-only emission (e.g. NeuralWatt
glm-5.3-flashor Openrouterglm-5.3-flash). - Run headless with a prompt that requires a withheld tool:
cmd -p "Search the web for 'z.ai GLM-5.3 flash release notes' and summarize in one sentence." \ --model <byok-provider>/glm-5.3-flash --output-format json --max-turns 6 - Observe the NDJSON event stream: turn 1 executes
search_toolsfine (declared tool), then subsequent turns produce text announcements or empty completions withcontinuation_recoveryevents, ending atmax_turnswithout the web search ever running. - Client-free repro (no Command Code involved): replay the captured failing request body directly against the provider endpoint; the same non-emission is deterministic. A ready-made script and trace bundle are available (
ab-matrix.mjs+ captured request bodies).
Command Code Version
1.40.0
Operating System
Linux
Terminal/IDE
Unknown
Shell
bash
Session file (optional)
No response
Fix prompt (optional)
No response
Additional context
- Wire evidence was captured with a local logging proxy between Command Code and the provider: full request bodies and raw SSE responses showing
finish_reason: "stop"with zerotool_callsdeltas despite in-stream reasoning explicitly planning the call. Traces available on request (auth headers redacted). - This is a contract gap rather than a one-sided bug: Anthropic-wire endpoints and lenient OpenAI-compatible servers pass
tool_callsthrough without validating against the request'stoolsarray, so progressive disclosure works there by luck. Strict implementations treattoolsas the authoritative emission set — a defensible reading of the spec and the safer training contract, since an undeclaredtool_callsentry is a hard error downstream. It will reproduce against any strictly-implemented OpenAI-wire provider, not just the one observed here. - Suggested fix: for
openai-completions-wire providers, declare all available tool schemas up front intools(or offer a per-provider "declare everything" mode), rather than relying on leniency that only some servers extend. Alternative: a compatibility flag users can set per provider inproviders.json. - Environment: Command Code CLI 26.7.0 on Linux, BYOK
openai-completionswire, streaming withstream_options.include_usage. The client parses tool-call deltas correctly (the declaredsearch_toolscall is parsed and executed in every run, including from the same failing session).
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Đánh giá
Issue này chưa được đánh giá.