antirez / antirez/ds4

[Feature request] GLM 5.3: server-side reasoning budget / forced think-close (repro data + working client-side pattern included)

Đang mở
#898 2 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

**Setup:** `glm-5.3-flash` branch @ a60a2a0, GLM-5.3-Flash-Q2.gguf (sha256 e81fd624…9705b32), M5 Max 128GB, `--ctx 393216 --warm-weights`, `glm-5.3-flash-reasoner` with `reasoning_effort: high`.

## Problem

On dense multi-requirement codegen specs, GLM 5.3 at High effort can think without bound and return an **empty reply**: `finish_reason: "length"`, `content: ""`, with the whole `max_tokens` spent inside ``. On the attached repro prompt (a 9-clause config-parser spec, ~2KB), **4 of 5 runs at max_tokens=60000 returned empty** (~223K–232K chars of reasoning_content each, ~40 min/run at ~25 t/s).

It is not a degeneration loop: shingle-repetition analysis of the reasoning shows zero verbatim repetition, and with max_tokens=120000 the model **finished at ~66K thinking tokens with fully correct code**. It's unbounded (but progressing) deliberation — there is just no way to bound it server-side today: effort levels are prompt prefixes, and neither `budget_tokens` (Anthropic `/v1/messages` `thinking` object) nor any `reasoning_max_tokens` knob is honored.

## Request

A server-side thinking budget that force-closes `` when N reasoning tokens are reached, so generation must produce the final answer. Suggested surface: honor `thinking: {type: "enabled", budget_tokens: N}` on `/v1/messages`, and/or a `reasoning_max_tokens` field on `/v1/chat/completions`.

## Working client-side pattern (validated — may be worth building in)

Two-phase, exploiting two properties of `render_glm_chat_prompt_text`:
1. an assistant message whose content starts with a think tag is rendered verbatim (`append_glm_assistant_message_prefix` early-return);
2. a trailing assistant turn gets no end-of-turn token, so generation continues it.

Phase 1: normal reasoner call with `max_tokens = budget`. On `finish_reason=length` + empty content, phase 2: resend with a trailing assistant message `"" + captured_reasoning + "\n\n[budget reached — committing to an answer]"` using the **nothink** model alias. On the repro spec, a 24K budget produced correct, test-passing code on 2/2 runs in ~40% of the unforced wall time (1123s vs 2838s); 16K terminated but degraded quality.

(Phase 2 must use the nothink alias due to a separate output-classification bug filed as #897: in reasoner mode the continued answer is misfiled into `reasoning_content`.)

Repro prompt and the two-phase script attached / available on request.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.