History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache
- Dominant language
- TypeScript
- Stars
- 5.4k
- Forks
- 502
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 715
Description
## What happened
Mid-turn history compaction never succeeds on the `kimi-coding-plan` / `k3-256k` connection, so long sessions grow past the context window and every agent-loop step slows down (10–30+ s per model call, request payload growing ~5 KB per step).
The capacity trigger itself works: the estimate anchors on real provider usage (~308K input tokens against a 262,144 window, high-water ≈ 245,760) and fires with reason `context_limit`. But every compaction attempt fails open with `failOpenReason: malformed_summary_missing_section` — in one send I counted **51 consecutive failed-open decisions, all for this reason**. Across the workspace there are **180 `history_compact` model calls on `k3-256k` and zero checkpoints** since 2026-08-20; the only historical checkpoints predate the sectioned-summary contract (#3029) and are legacy free-form text.
Consequences observed in production data:
- Sessions run at 250K–315K input tokens per step (~900 KB request payloads); TTFT 6–12 s, per-step latency 10–30+ s even with ~99% prefix-cache hits.
- Every turn wastes 1–2 summarizer calls that are guaranteed to be discarded (see root cause), and those calls are themselves slow: 8–43 s typical, up to **88 s** (see cache section below).
- Provider errors are not circuit-broken: one session logged **15 consecutive failed summarizer calls over ~47 minutes** before its main request.
中文
在 `kimi-coding-plan` / `k3-256k` 连接上,mid-turn 历史压缩从未成功过。长会话因此不断膨胀直至超过上下文窗口,agent loop 每一步都变慢(每次模型调用 10–30+ 秒,请求体每步增长约 5 KB)。
容量触发逻辑本身是正常的:估算锚定在真实 provider usage 上(约 308K input tokens,窗口 262,144,high-water ≈ 245,760),并以 `context_limit` 原因触发。但每次压缩尝试都以 `failOpenReason: malformed_summary_missing_section` 失败放行——单次 send 里有 **51 条连续 failed-open 决策,全是这个原因**。整个 workspace 里,k3-256k 上有 **180 次 `history_compact` 调用、零条 checkpoint**(自 2026-08-20 起);仅有的历史 checkpoint 都早于分节摘要契约(#3029),是遗留的自由文本。
生产数据中观察到的后果:
- 会话每步携带 250K–315K input tokens(请求体约 900 KB);即使前缀缓存命中率约 99%,TTFT 仍达 6–12 秒,单步延迟 10–30+ 秒。
- 每个 turn 都会浪费 1–2 次注定被丢弃的 summarizer 调用(见根因),这些调用本身也很慢:通常 8–43 秒,最高 **88 秒**(见下文缓存部分)。
- Provider 错误没有熔断:一个会话在约 47 分钟内记录了 **15 次连续失败的 summarizer 调用**,主请求被拖住。
### Root cause: `k3-256k` ignores the summarization instruction when it is a system prompt
`buildLlmHistorySummarizer` sends `SUMMARIZATION_SYSTEM_PROMPT` as the system prompt followed by the conversation. `k3-256k` is an agentic coding model (reasoning always on, default effort `max`); in this shape it does not switch into summarizer mode — it continues the conversation as the assistant, returning a stub (`content: null` / 7–72 output tokens, `end_turn`) that can never satisfy the required `## Goal / ## Progress / ## Next Steps / ## Critical Context` sections. The #3029 validator is working as designed; the request shape is incompatible with this model.
Reproduced against the production endpoint (`api.kimi.com/coding`, same key) with a **367-token toy conversation**, so input size is not the trigger:
| Request shape | k3-256k result |
|---|---|
| Anthropic wire, instruction in `system` field (maka's current shape) | `content: null`, 7 output tokens, `stop_reason: end_turn` — no summary at all |
| OpenAI-chat wire, instruction as system message | Ignores the format, answers the user's question as the assistant (1,567 tokens, no section headings) |
| Either wire, **identical instruction as a trailing user message** after the conversation | Fully compliant sectioned summary (all 4 required sections, in order, substantive content) |
This also explains why the same account/model compacts fine in other agents (e.g. OpenCode): they deliver the compaction instruction as a user message at the end of the conversation, not as a system prompt.
中文
`buildLlmHistorySummarizer` 把 `SUMMARIZATION_SYSTEM_PROMPT` 放在 system prompt、对话跟在后面。`k3-256k` 是 agentic coding 模型(reasoning 恒开、默认 effort=max);在这种形态下它不会切换成"摘要器"角色,而是继续以助手身份延续对话,返回残缺输出(`content: null` / 7–72 output tokens,`end_turn`),永远凑不齐必需的四个小节(`## Goal / ## Progress / ## Next Steps / ## Critical Context`)。#3029 的校验器工作正常;问题出在请求形态与这个模型不兼容。
用同一 key 对生产端点(`api.kimi.com/coding`)以 **367-token 的玩具对话**复现,因此与输入大小无关(表格见上方英文版)。
这也解释了为什么同一个账号/模型在其他 agent(如 OpenCode)里压缩正常:它们把压缩指令放在对话末尾的 user 消息里,而不是 system prompt。
### Side effect: the system-prompt swap also destroys prefix caching for summarizer calls
Because the summarizer request starts with a *different* system prompt, it shares no prefix with the main-loop requests and pays a full uncached pass over the entire conversation. Observed:
- Main calls: cacheRead ≈ 316K/322K input (99.8% hit), 11–33 s.
- `history_compact` first attempts: `cacheReadInputTokens: 0` on 155K–253K inputs → 39 s, 65 s, **88 s** per wasted call. (The repair retry immediately after hits the first attempt's fresh cache: 180,736/181,894 cache-read, 17 s.)
Delivering the instruction as a trailing user message would make the summarizer request share the main loop's whole cached prefix (~99% hits on the first attempt too), so the fix for instruction-following also fixes this.
中文
由于 summarizer 请求以一个*不同的* system prompt 开头,它与主循环请求没有任何共享前缀,整段对话都要全量冷处理。实测:主调用 cacheRead ≈ 316K/322K(99.8% 命中),而 `history_compact` 首次尝试在 155K–253K 输入上 `cacheReadInputTokens: 0`,每次白跑的调用耗时 39–**88 秒**。
如果把指令放到对话末尾的 user 消息,summarizer 请求就能共享主循环的整段缓存前缀(首次尝试也有 ~99% 命中)——修指令遵循问题的同时也修好了缓存问题。
### Contributing defects found while tracing
1. **The summarizer never sets `maxOutputTokens`** — `generateSummary` calls `generateText` without it, while main-loop requests resolve 131,072 via `selectedModelMaxOutputTokens`. On reasoning models an unset output budget is at best unpredictable.
2. **The failure circuit only covers malformed-summary reasons.** `summarizeWithFailureCircuit` records only `isMalformedHistoryCompactSummaryReason`, and per-send `state.malformedSummaryFailure` latches only those too; `provider_error` / `output_length` therefore retry on *every* step — this produced the 15-call retry storm above.
3. **The summarizer input budget is enormous** (`capacity − reserve` ≈ 245K tokens), so a fold at high context feeds ~180–235K tokens into the summarizer — expensive, slow, and further degrades compliance, feeding the vicious cycle.
中文
1. **summarizer 从不设置 `maxOutputTokens`**——`generateSummary` 调 `generateText` 时不传,而主循环请求会通过 `selectedModelMaxOutputTokens` 解析出 131,072。对 reasoning 模型,不设输出预算的行为难以预期。
2. **熔断只覆盖摘要格式错误**:`summarizeWithFailureCircuit` 只记录 `isMalformedHistoryCompactSummaryReason`,send 级的 `state.malformedSummaryFailure` 也只锁存这类原因;`provider_error` / `output_length` 因此**每一步**都会重试——上面的 15 次重试风暴就是这么来的。
3. **summarizer 输入预算过大**(`capacity − reserve` ≈ 245K tokens),高上下文时一次 fold 会喂给 summarizer 约 180–235K tokens——又贵又慢,还进一步降低指令遵循度,加剧恶性循环。
## How to reproduce
1. Run a session on `kimi-coding-plan` / `k3-256k` until it approaches the high-water mark.
2. Inspect the `token_usage` message diagnostics: `contextBudget.compactionDecisions` fills with `{stage: activeStep, decision: failedOpen, phase: mid_turn, reason: context_limit, failOpenReason: malformed_summary_missing_section}`.
3. `model_call_attempt_recorded` events with `callKind: history_compact` show `outputTokens` of 40–72 with `finishReason: end_turn` — far too short to contain the four required sections.
4. Minimal API-level repro: POST to `https://api.kimi.com/coding/v1/messages` with model `k3-256k`, `system` = `SUMMARIZATION_SYSTEM_PROMPT`, and any short user/assistant exchange → the model returns `content: null` / a few tokens instead of the sectioned summary. Moving the same instruction into a trailing user message returns a compliant summary.
中文
1. 在 `kimi-coding-plan` / `k3-256k` 上跑一个会话直到接近 high-water。
2. 查看 `token_usage` 消息的 diagnostics:`contextBudget.compactionDecisions` 会被 `{stage: activeStep, decision: failedOpen, phase: mid_turn, reason: context_limit, failOpenReason: malformed_summary_missing_section}` 填满。
3. `callKind: history_compact` 的 `model_call_attempt_recorded` 事件显示 `outputTokens` 只有 40–72、`finishReason: end_turn`——太短,不可能包含四个必需小节。
4. 最小 API 级复现:向 `https://api.kimi.com/coding/v1/messages` POST,model 为 `k3-256k`,`system` = `SUMMARIZATION_SYSTEM_PROMPT`,加任意短对话 → 模型返回 `content: null` / 几个 token 而非分节摘要。把同样的指令移到末尾 user 消息则返回合规摘要。
## Environment
- Maka commit: `898b86d6b7` (source build)
- OS: Linux 7.0.0-30-generic x86_64
- Surface: Runtime Host / TUI
- Node.js: v26.3.0
- Connection: `kimi-coding-plan`, model `k3-256k` (contextWindow 262144, reasoning always on, anthropic-messages wire)
## Logs, screenshots, or additional context
Directions I'd like maintainer input on (happy to split into separate issues/PRs):
1. **Deliver the summarization instruction as a trailing user message** after the conversation (verified working on both wires for k3-256k, and restores prefix-cache reuse) — smallest change, in `buildLlmHistorySummarizer`. Open question: is the system-prompt placement load-bearing for other providers, or can this be unconditional?
2. Pass an explicit, bounded `maxOutputTokens` for summarizer calls.
3. Extend the failure circuit to `provider_error` / `output_length` so provider-side failures don't retry every step.
4. Bound the summarizer input budget well below `capacity − reserve` (e.g. fold in increments via the existing rolling `previousCheckpoint` mechanism).
Evidence is from the local workspace store: `runtime.sqlite` (`core_agent_run_events`, `usage_model_call_attempts`, `token_usage` diagnostics) and per-request capture artifacts. I can share redacted excerpts if useful.
中文
希望听取维护者意见的方向(可随时拆成独立 issue/PR):
1. **把摘要指令放到对话末尾的 user 消息**(已在 k3-256k 的两条 wire 上验证有效,且能恢复前缀缓存复用)——改动最小,位于 `buildLlmHistorySummarizer`。待讨论:system prompt 形态对其他 provider 是否有依赖,还是可以无条件改?
2. 为 summarizer 调用显式传入有界的 `maxOutputTokens`。
3. 把熔断扩展到 `provider_error` / `output_length`,避免 provider 侧故障时每步重试。
4. 把 summarizer 输入预算压到远低于 `capacity − reserve`(例如借助现有的滚动 `previousCheckpoint` 机制做增量折叠)。
证据来自本地 workspace 存储:`runtime.sqlite`(`core_agent_run_events`、`usage_model_call_attempts`、`token_usage` diagnostics)以及逐请求捕获的 artifact。需要的话我可以提供脱敏摘录。
Contributor guide
Assessment
This issue has not been assessed yet.