MoonshotAI / MoonshotAI/kimi-code

[Bug] K2.7-Code on self-hosted OpenAI-compatible endpoint: model intermittently ends turn after thinking (finish=stop, no text/tool call) in multi-agent runs — APIEmptyResponseError

Open
#3,429 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
1.2k
Avg merge
11h 53m
Merged PRs (30d)
350

Description

Summary

Running Kimi-K2.7-Code on a self-hosted OpenAI-compatible endpoint (vLLM 0.19.1, --tool-call-parser kimi_k2 --reasoning-parser kimi_k2), multi-agent workloads (AgentSwarm, 12 sub-agents) intermittently fail with APIEmptyResponseError. Extracting finishReason from every failing retry shows the dominant failure is the model emitting thinking and then a normal stop token — no text, no tool call is ever generated:

failing retries by finishReason, per run (kimi-code 0.39.1):
  7x completed   (vLLM 0.19.1)
  9x completed   (vLLM 0.19.1 + DCP)
  9x completed   (vLLM 0.28.0)
  2x tool_calls  (separate vLLM streaming-parser bug, filed as vllm-project/vllm#54701)

The finishReason=completed cases are engine-independent (reproduced identically on vLLM 0.19.1 and 0.28.0), so this is not the server's streaming parser — the model ends its turn after the reasoning block. Recorded wire-log content parts for a failing step look like:

{"part":{"type":"think","think":" One agent came back empty, so I'll quickly read that file myself and include it in the merged summary."}}
→ (no text part, no tool.call part; turn retried with APIEmptyResponseError)

i.e. the reasoning concludes with a clear intent to act, then the model stops. Retries hit the same behavior often enough that sub-agents exhaust attempts (typical damage: 1–4 failed sub-agents per 12-agent swarm; ~5–30% of tool-call-bearing requests under concurrent load; never observed single-stream).

What we tested (both negative)
  1. System-prompt nudge ("after thinking, always emit the tool call or text — never end on thinking alone", injected via workspace AGENTS.md, confirmed present in sub-agent contexts): no improvement (9/12 and 8/12 completions vs the same baseline band).
  2. KIMI_MODEL_THINKING_KEEP=off (verified to bind — thinkingKeep disappears from the CLI's request records): one 12/12 clean run, then 10/12 with 8× the same finishReason=completed failures — within run-to-run noise, not a fix.
Questions / ask
  • Is think-only-then-stop a known K2.7-Code behavior in multi-turn agentic contexts where prior-turn reasoning is replayed (thinkingKeep: all default) to an OpenAI-compatible endpoint?
  • Is the CLI's reasoning replay format for type = "openai" providers (assistant reasoning field) what the model's chat template expects, or should replayed thinking be rendered differently for self-hosted vLLM?
  • Any recommended serving-side or CLI-side setting to prevent the model ending a turn inside/at the end of reasoning?
Environment
  • kimi-code CLI 0.39.1, provider type = "openai", model entry: max_context_size 131072, max_output_size 32768, capabilities ["thinking","always_thinking","tool_use"], thinking effort high
  • Server: vLLM 0.19.1 (also 0.28.0), moonshotai/Kimi-K2.7-Code official INT4, TP=8, --enable-auto-tool-choice --tool-call-parser kimi_k2 --reasoning-parser kimi_k2, temp 1.0 / top_p 0.95
  • Full wire logs (CLI session records) and SSE captures available on request.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with the listed kimi-code CLI, OpenAI-compatible provider, vLLM versions, and multi-agent workload. Inspect the CLI session records and available wire logs, focusing on reasoning replay, thinkingKeep, finishReason=completed, and empty responses. Done means isolating whether the CLI replay format, model behavior, or serving configuration causes the think-only stop and documenting a narrowly scoped fix or confirmed limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.