antirez / antirez/ds4

[Feature request] GLM 5.3: server-side reasoning budget / forced think-close (repro data + working client-side pattern included)

Open
#898 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.3k
Forks
2.1k
Avg merge
1d 3h
Merged PRs (30d)
4

Description

**Setup:** `glm-5.3-flash` branch @ a60a2a0, GLM-5.3-Flash-Q2.gguf (sha256 e81fd624…9705b32), M5 Max 128GB, `--ctx 393216 --warm-weights`, `glm-5.3-flash-reasoner` with `reasoning_effort: high`.

## Problem

On dense multi-requirement codegen specs, GLM 5.3 at High effort can think without bound and return an **empty reply**: `finish_reason: "length"`, `content: ""`, with the whole `max_tokens` spent inside ``. On the attached repro prompt (a 9-clause config-parser spec, ~2KB), **4 of 5 runs at max_tokens=60000 returned empty** (~223K–232K chars of reasoning_content each, ~40 min/run at ~25 t/s).

It is not a degeneration loop: shingle-repetition analysis of the reasoning shows zero verbatim repetition, and with max_tokens=120000 the model **finished at ~66K thinking tokens with fully correct code**. It's unbounded (but progressing) deliberation — there is just no way to bound it server-side today: effort levels are prompt prefixes, and neither `budget_tokens` (Anthropic `/v1/messages` `thinking` object) nor any `reasoning_max_tokens` knob is honored.

## Request

A server-side thinking budget that force-closes `` when N reasoning tokens are reached, so generation must produce the final answer. Suggested surface: honor `thinking: {type: "enabled", budget_tokens: N}` on `/v1/messages`, and/or a `reasoning_max_tokens` field on `/v1/chat/completions`.

## Working client-side pattern (validated — may be worth building in)

Two-phase, exploiting two properties of `render_glm_chat_prompt_text`:
1. an assistant message whose content starts with a think tag is rendered verbatim (`append_glm_assistant_message_prefix` early-return);
2. a trailing assistant turn gets no end-of-turn token, so generation continues it.

Phase 1: normal reasoner call with `max_tokens = budget`. On `finish_reason=length` + empty content, phase 2: resend with a trailing assistant message `"" + captured_reasoning + "\n\n[budget reached — committing to an answer]"` using the **nothink** model alias. On the repro spec, a 24K budget produced correct, test-passing code on 2/2 runs in ~40% of the unforced wall time (1123s vs 2838s); 16K terminated but degraded quality.

(Phase 2 must use the nothink alias due to a separate output-classification bug filed as #897: in reasoner mode the continued answer is misfiled into `reasoning_content`.)

Repro prompt and the two-phase script attached / available on request.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.