antirez / antirez/ds4

Context Window Exhaustion Causes Unterminated Tool Calls

未关闭
#48 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C
星标
22.3k
派生
2.1k
平均合并
1 天 3 小时
30 天内合并 PR
4

描述

Symptom
When accumulated conversation fills the context window (ctx=32768), the model occasionally starts a <|DSML|tool_calls> block but fails to close it before hitting the context limit. The server then rejects the generation:

ctx=22528..31488:8960 gen=1280 DSML_START finish=error error="unterminated tool call"
Total context position at exit: 31488 + 1280 = 32768 — exactly the ctx hard limit.

What happens in code
The decode while loop at [ds4_server.c:5409](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5409) exits when ds4_session_pos >= ds4_session_ctx.
The post-loop safety check at [ds4_server.c:5583-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5583-L5589) (added in 404f0a1) detects saw_tool_start && !saw_tool_end and flags it as an error.
The user gets a failure with no clear signal about what went wrong.
Why it happens
Tool calls in ds4 are represented as plain text strings (<|DSML|tool_calls>...). The server detects them via post-hoc strstr() scanning on the generated text ([ds4_server.c:5525-5533](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5525-L5533)). The model samples tokens freely with zero awareness of its remaining context budget. When it starts a tool call late in the generation, nothing prevents it from running out of room before writing the closing tag.

Affected code paths
[ds4_server.c:5390-5574](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5390-L5574) — main decode loop, context budget gating
[ds4_server.c:5523-5534](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5523-L5534) — text-based tool_calls_started / tool_calls_finished detection
[ds4_server.c:5582-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5582-L5589) — unterminated tool call rejection

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。