CommandCodeAI / CommandCodeAI/command-code
Tool calls silently fail on strict OpenAI-wire BYOK providers: web_search/MCP tools announced in text or empty completion instead of tool_use — undeclared-tool progressive disclosure not tolerated by all servers
@ahmadbilaldev is already working on this.
Since Sep 17, 2026.
- Dominant language
- No language data
- Stars
- 4k
- Forks
- 350
- PR merge metrics
- No merged PRs in 30d
Description
Summary
Command Code uses progressive tool disclosure: most tool schemas (web_search, web_fetch, all mcp__* tools) are not declared in the request's tools array. Their schemas are delivered lazily as the text result of a search_tools call, and the model is expected to emit a tool_calls block for a tool that is not present in the OpenAI tools parameter of that request.
Some OpenAI-wire serving stacks tolerate this (emitting tool_calls for undeclared, in-prompt-described tools); others — in our case NeuralWatt's GLM serving (glm-5.3-flash, glm-5.2) and Openrouter — strictly constrain emission to declared tools. On those servers the model either writes a text reply announcing the call ("Now I'll run the web search…") with finish_reason: "stop" and zero tool_calls deltas, or returns a completely empty completion. The harness's continuation recovery then burns turns retrying, and the run ends with the announcement as finalText without ever executing the tool.
The failure was reproduced client-free: replaying the exact captured request body directly to the provider with curl/a raw script produces the same non-emission 100% of the time (0/10), and adding the tool to the request's tools array fixes it 100% of the time (10/10) with everything else identical.
Expected Behavior
The model should emit a tool_calls block for web_search (and MCP tools) after their schemas have been loaded via search_tools, the harness should execute them, and the run should complete normally — as it does on providers that tolerate undeclared-tool calling.
Actual Behavior
- Assistant streams reasoning planning the tool call, then streams text announcing it ("Now I'll run the web search for…") and the turn ends with
finish_reason: "stop"— notool_callsdeltas in the stream. - Sometimes the completion is entirely empty (no content, no reasoning) with
finish_reason: "stop". - The harness fires
continuation_recovery(kind: "empty"/kind: "intent") and retries; failures are sticky within a session and the run often ends atmax_turnswith the announcement asfinalText. - Only tools whose schemas are declared in
tools(e.g.search_tools,shell_command,read_file) work reliably, matching the "simple tools work, web/MCP tools don't" symptom. - A/B replay matrix on the captured failing payload (5 repeats per variant, both
glm-5.3-flashandglm-5.2):- Captured payload as-is → 0/10
tool_calls(text-announce or empty) - Same payload with continuation nudge removed → 0/10 (all empty)
- Same payload +
web_searchdeclared intools→ 10/10tool_calls - Minimal payload (same system+user, only
web_searchtool) → 10/10tool_calls
- Captured payload as-is → 0/10
Context size is ruled out: the minimal-payload variant succeeds even though the failing payload carries the full ~96 KB harness context; the deciding factor is solely whether the tool is declared in tools.
Steps to reproduce the issue
- Attach any MCP server and use a BYOK provider on the OpenAI-completions wire whose serving enforces declared-tool-only emission (e.g. NeuralWatt
glm-5.3-flashor Openrouterglm-5.3-flash). - Run headless with a prompt that requires a withheld tool:
cmd -p "Search the web for 'z.ai GLM-5.3 flash release notes' and summarize in one sentence." \ --model <byok-provider>/glm-5.3-flash --output-format json --max-turns 6 - Observe the NDJSON event stream: turn 1 executes
search_toolsfine (declared tool), then subsequent turns produce text announcements or empty completions withcontinuation_recoveryevents, ending atmax_turnswithout the web search ever running. - Client-free repro (no Command Code involved): replay the captured failing request body directly against the provider endpoint; the same non-emission is deterministic. A ready-made script and trace bundle are available (
ab-matrix.mjs+ captured request bodies).
Command Code Version
1.40.0
Operating System
Linux
Terminal/IDE
Unknown
Shell
bash
Session file (optional)
No response
Fix prompt (optional)
No response
Additional context
- Wire evidence was captured with a local logging proxy between Command Code and the provider: full request bodies and raw SSE responses showing
finish_reason: "stop"with zerotool_callsdeltas despite in-stream reasoning explicitly planning the call. Traces available on request (auth headers redacted). - This is a contract gap rather than a one-sided bug: Anthropic-wire endpoints and lenient OpenAI-compatible servers pass
tool_callsthrough without validating against the request'stoolsarray, so progressive disclosure works there by luck. Strict implementations treattoolsas the authoritative emission set — a defensible reading of the spec and the safer training contract, since an undeclaredtool_callsentry is a hard error downstream. It will reproduce against any strictly-implemented OpenAI-wire provider, not just the one observed here. - Suggested fix: for
openai-completions-wire providers, declare all available tool schemas up front intools(or offer a per-provider "declare everything" mode), rather than relying on leniency that only some servers extend. Alternative: a compatibility flag users can set per provider inproviders.json. - Environment: Command Code CLI 26.7.0 on Linux, BYOK
openai-completionswire, streaming withstream_options.include_usage. The client parses tool-call deltas correctly (the declaredsearch_toolscall is parsed and executed in every run, including from the same failing session).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.