[Feature request] GLM 5.3: server-side reasoning budget / forced think-close (repro data + working client-side pattern included)
- Langage dominant
- C
- Étoiles
- 22.3k
- Forks
- 2.1k
- Merge moyen
- 1 j 3 h
- PR mergées (30 j)
- 4
Description
**Setup:** `glm-5.3-flash` branch @ a60a2a0, GLM-5.3-Flash-Q2.gguf (sha256 e81fd624…9705b32), M5 Max 128GB, `--ctx 393216 --warm-weights`, `glm-5.3-flash-reasoner` with `reasoning_effort: high`.
## Problem
On dense multi-requirement codegen specs, GLM 5.3 at High effort can think without bound and return an **empty reply**: `finish_reason: "length"`, `content: ""`, with the whole `max_tokens` spent inside ``. On the attached repro prompt (a 9-clause config-parser spec, ~2KB), **4 of 5 runs at max_tokens=60000 returned empty** (~223K–232K chars of reasoning_content each, ~40 min/run at ~25 t/s).
It is not a degeneration loop: shingle-repetition analysis of the reasoning shows zero verbatim repetition, and with max_tokens=120000 the model **finished at ~66K thinking tokens with fully correct code**. It's unbounded (but progressing) deliberation — there is just no way to bound it server-side today: effort levels are prompt prefixes, and neither `budget_tokens` (Anthropic `/v1/messages` `thinking` object) nor any `reasoning_max_tokens` knob is honored.
## Request
A server-side thinking budget that force-closes `` when N reasoning tokens are reached, so generation must produce the final answer. Suggested surface: honor `thinking: {type: "enabled", budget_tokens: N}` on `/v1/messages`, and/or a `reasoning_max_tokens` field on `/v1/chat/completions`.
## Working client-side pattern (validated — may be worth building in)
Two-phase, exploiting two properties of `render_glm_chat_prompt_text`:
1. an assistant message whose content starts with a think tag is rendered verbatim (`append_glm_assistant_message_prefix` early-return);
2. a trailing assistant turn gets no end-of-turn token, so generation continues it.
Phase 1: normal reasoner call with `max_tokens = budget`. On `finish_reason=length` + empty content, phase 2: resend with a trailing assistant message `"" + captured_reasoning + "\n\n[budget reached — committing to an answer]"` using the **nothink** model alias. On the repro spec, a 24K budget produced correct, test-passing code on 2/2 runs in ~40% of the unforced wall time (1123s vs 2838s); 16K terminated but degraded quality.
(Phase 2 must use the nothink alias due to a separate output-classification bug filed as #897: in reasoner mode the continued answer is misfiled into `reasoning_content`.)
Repro prompt and the two-phase script attached / available on request.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.