antirez / antirez/ds4

Context Window Exhaustion Causes Unterminated Tool Calls

Đang mở
#48 1 bình luận 1 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

Symptom
When accumulated conversation fills the context window (ctx=32768), the model occasionally starts a <|DSML|tool_calls> block but fails to close it before hitting the context limit. The server then rejects the generation:

ctx=22528..31488:8960 gen=1280 DSML_START finish=error error="unterminated tool call"
Total context position at exit: 31488 + 1280 = 32768 — exactly the ctx hard limit.

What happens in code
The decode while loop at [ds4_server.c:5409](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5409) exits when ds4_session_pos >= ds4_session_ctx.
The post-loop safety check at [ds4_server.c:5583-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5583-L5589) (added in 404f0a1) detects saw_tool_start && !saw_tool_end and flags it as an error.
The user gets a failure with no clear signal about what went wrong.
Why it happens
Tool calls in ds4 are represented as plain text strings (<|DSML|tool_calls>...). The server detects them via post-hoc strstr() scanning on the generated text ([ds4_server.c:5525-5533](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5525-L5533)). The model samples tokens freely with zero awareness of its remaining context budget. When it starts a tool call late in the generation, nothing prevents it from running out of room before writing the closing tag.

Affected code paths
[ds4_server.c:5390-5574](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5390-L5574) — main decode loop, context budget gating
[ds4_server.c:5523-5534](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5523-L5534) — text-based tool_calls_started / tool_calls_finished detection
[ds4_server.c:5582-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5582-L5589) — unterminated tool call rejection

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.