MoonshotAI / MoonshotAI/kimi-code
[Bug] K2.7-Code on self-hosted OpenAI-compatible endpoint: model intermittently ends turn after thinking (finish=stop, no text/tool call) in multi-agent runs — APIEmptyResponseError
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 11h 53m
- Merged PRs (30d)
- 350
Description
Summary
Running Kimi-K2.7-Code on a self-hosted OpenAI-compatible endpoint (vLLM 0.19.1, --tool-call-parser kimi_k2 --reasoning-parser kimi_k2), multi-agent workloads (AgentSwarm, 12 sub-agents) intermittently fail with APIEmptyResponseError. Extracting finishReason from every failing retry shows the dominant failure is the model emitting thinking and then a normal stop token — no text, no tool call is ever generated:
failing retries by finishReason, per run (kimi-code 0.39.1):
7x completed (vLLM 0.19.1)
9x completed (vLLM 0.19.1 + DCP)
9x completed (vLLM 0.28.0)
2x tool_calls (separate vLLM streaming-parser bug, filed as vllm-project/vllm#54701)
The finishReason=completed cases are engine-independent (reproduced identically on vLLM 0.19.1 and 0.28.0), so this is not the server's streaming parser — the model ends its turn after the reasoning block. Recorded wire-log content parts for a failing step look like:
{"part":{"type":"think","think":" One agent came back empty, so I'll quickly read that file myself and include it in the merged summary."}}
→ (no text part, no tool.call part; turn retried with APIEmptyResponseError)
i.e. the reasoning concludes with a clear intent to act, then the model stops. Retries hit the same behavior often enough that sub-agents exhaust attempts (typical damage: 1–4 failed sub-agents per 12-agent swarm; ~5–30% of tool-call-bearing requests under concurrent load; never observed single-stream).
What we tested (both negative)
- System-prompt nudge ("after thinking, always emit the tool call or text — never end on thinking alone", injected via workspace AGENTS.md, confirmed present in sub-agent contexts): no improvement (9/12 and 8/12 completions vs the same baseline band).
KIMI_MODEL_THINKING_KEEP=off(verified to bind —thinkingKeepdisappears from the CLI's request records): one 12/12 clean run, then 10/12 with 8× the samefinishReason=completedfailures — within run-to-run noise, not a fix.
Questions / ask
- Is think-only-then-stop a known K2.7-Code behavior in multi-turn agentic contexts where prior-turn reasoning is replayed (
thinkingKeep: alldefault) to an OpenAI-compatible endpoint? - Is the CLI's reasoning replay format for
type = "openai"providers (assistantreasoningfield) what the model's chat template expects, or should replayed thinking be rendered differently for self-hosted vLLM? - Any recommended serving-side or CLI-side setting to prevent the model ending a turn inside/at the end of reasoning?
Environment
- kimi-code CLI 0.39.1, provider
type = "openai", model entry:max_context_size 131072,max_output_size 32768, capabilities["thinking","always_thinking","tool_use"], thinking effort high - Server: vLLM 0.19.1 (also 0.28.0),
moonshotai/Kimi-K2.7-Codeofficial INT4, TP=8,--enable-auto-tool-choice --tool-call-parser kimi_k2 --reasoning-parser kimi_k2, temp 1.0 / top_p 0.95 - Full wire logs (CLI session records) and SSE captures available on request.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with the listed kimi-code CLI, OpenAI-compatible provider, vLLM versions, and multi-agent workload. Inspect the CLI session records and available wire logs, focusing on reasoning replay, thinkingKeep, finishReason=completed, and empty responses. Done means isolating whether the CLI replay format, model behavior, or serving configuration causes the think-only stop and documenting a narrowly scoped fix or confirmed limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100