antirez / antirez/ds4

Context Window Exhaustion Causes Unterminated Tool Calls

Abierto
#48 1 comentario 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

Symptom
When accumulated conversation fills the context window (ctx=32768), the model occasionally starts a <|DSML|tool_calls> block but fails to close it before hitting the context limit. The server then rejects the generation:

ctx=22528..31488:8960 gen=1280 DSML_START finish=error error="unterminated tool call"
Total context position at exit: 31488 + 1280 = 32768 — exactly the ctx hard limit.

What happens in code
The decode while loop at [ds4_server.c:5409](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5409) exits when ds4_session_pos >= ds4_session_ctx.
The post-loop safety check at [ds4_server.c:5583-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5583-L5589) (added in 404f0a1) detects saw_tool_start && !saw_tool_end and flags it as an error.
The user gets a failure with no clear signal about what went wrong.
Why it happens
Tool calls in ds4 are represented as plain text strings (<|DSML|tool_calls>...). The server detects them via post-hoc strstr() scanning on the generated text ([ds4_server.c:5525-5533](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5525-L5533)). The model samples tokens freely with zero awareness of its remaining context budget. When it starts a tool call late in the generation, nothing prevents it from running out of room before writing the closing tag.

Affected code paths
[ds4_server.c:5390-5574](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5390-L5574) — main decode loop, context budget gating
[ds4_server.c:5523-5534](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5523-L5534) — text-based tool_calls_started / tool_calls_finished detection
[ds4_server.c:5582-5589](vscode-webview://1qlm686t3cfu6h06kuuhtltepirvsgfa2ning82l39it89jk635d/ds4_server.c#L5582-L5589) — unterminated tool call rejection

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.