ds4-server: no reply-budget or forced think-close, so HTTP callers can hang where ds4-eval terminates
- Vorherrschende Sprache
- C
- Sterne
- 22.3k
- Forks
- 2.1k
- Ø Merge
- 1 T. 3 Std.
- Gemergte PRs (30 T.)
- 4
Beschreibung
## Summary
`ds4-eval` can force a `` close as the reply budget runs out. `ds4-server` cannot, and exposes no equivalent. The same model, on the same saturating question, terminates through `ds4-eval` and does not terminate over HTTP. The difference is the call path, not the model.
`ds4_eval.c:1216-1218`, with defaults at 1522-1524 and CLI flags at 1578-1580:
```c
int soft_limit_reply_budget; // 1024
int hard_limit_reply_budget; // 512
int soft_limit_think_close_rank; // 3
```
`grep -c "reply_budget\|think_close_rank" ds4_server.c` returns 0, and `./ds4-server --help` lists no equivalent.
## Measured cost
Overnight run, GLM-5.3-Flash Q2 on an M3 Max 128 GB, `--ctx 393216`, 14 hours, 415,425 tokens generated.
`ds4-eval` scored 71/92 with **21 failures that are all wrong answers and no non-answers** — the forced close means every question terminates. One question reached exactly 16,000 tokens, the cap, and still graded `PASSED`, because the harness made it conclude.
A separate 16-question suite driven over the HTTP API scored 14/16. The two failures are non-terminations: 60 minutes of silence each, no output. Across the night, **96,868 tokens — 23% of everything generated — produced zero usable characters**, all of it on the HTTP path.
Length is also a usable failure signal on this model. Separating passed from failed AIME items:
| | mean problem number | mean tokens generated |
|---|---|---|
| passed | 11.6 | 2,484 |
| failed | 14.5 | 8,477 |
The declared difficulty explains nothing — the problem numbers are nearly identical — while length separates the two populations by 3.4x, and half the AIME failures are exact 16,000-token saturations. Correlation rather than causation, since a hard question produces both length and error, but the counter is available in real time either way.
## Why it matters beyond a benchmark
Any agent talking to `ds4-server` over HTTP is exposed to non-termination, including `ds4-agent` itself when it goes through the API. On this model the budget is not spent on work, it is spent on persistence.
## Proposal
Port the three settings from `ds4_eval.c` to `ds4_server.c`, with the same defaults, as server flags and optionally as per-request fields. The logic already exists and is exercised; this is a plumbing change rather than a new mechanism.
Related: #879 asks for `--nothink` on `ds4-server`, which is the same shape of gap — the CLI and the eval harness expose generation controls that the server does not.
Happy to provide the traces, or to test a patch on the same 128 GB M3 Max setup.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.