antirez / antirez/ds4

[Feature request] GLM 5.3: server-side reasoning budget / forced think-close (repro data + working client-side pattern included)

Abierto
#898 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

**Setup:** `glm-5.3-flash` branch @ a60a2a0, GLM-5.3-Flash-Q2.gguf (sha256 e81fd624…9705b32), M5 Max 128GB, `--ctx 393216 --warm-weights`, `glm-5.3-flash-reasoner` with `reasoning_effort: high`.

## Problem

On dense multi-requirement codegen specs, GLM 5.3 at High effort can think without bound and return an **empty reply**: `finish_reason: "length"`, `content: ""`, with the whole `max_tokens` spent inside ``. On the attached repro prompt (a 9-clause config-parser spec, ~2KB), **4 of 5 runs at max_tokens=60000 returned empty** (~223K–232K chars of reasoning_content each, ~40 min/run at ~25 t/s).

It is not a degeneration loop: shingle-repetition analysis of the reasoning shows zero verbatim repetition, and with max_tokens=120000 the model **finished at ~66K thinking tokens with fully correct code**. It's unbounded (but progressing) deliberation — there is just no way to bound it server-side today: effort levels are prompt prefixes, and neither `budget_tokens` (Anthropic `/v1/messages` `thinking` object) nor any `reasoning_max_tokens` knob is honored.

## Request

A server-side thinking budget that force-closes `` when N reasoning tokens are reached, so generation must produce the final answer. Suggested surface: honor `thinking: {type: "enabled", budget_tokens: N}` on `/v1/messages`, and/or a `reasoning_max_tokens` field on `/v1/chat/completions`.

## Working client-side pattern (validated — may be worth building in)

Two-phase, exploiting two properties of `render_glm_chat_prompt_text`:
1. an assistant message whose content starts with a think tag is rendered verbatim (`append_glm_assistant_message_prefix` early-return);
2. a trailing assistant turn gets no end-of-turn token, so generation continues it.

Phase 1: normal reasoner call with `max_tokens = budget`. On `finish_reason=length` + empty content, phase 2: resend with a trailing assistant message `"" + captured_reasoning + "\n\n[budget reached — committing to an answer]"` using the **nothink** model alias. On the repro spec, a 24K budget produced correct, test-passing code on 2/2 runs in ~40% of the unforced wall time (1123s vs 2838s); 16K terminated but degraded quality.

(Phase 2 must use the nothink alias due to a separate output-classification bug filed as #897: in reasoner mode the continued answer is misfiled into `reasoning_content`.)

Repro prompt and the two-phase script attached / available on request.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.