Model reaches correct answer internally but won't commit — repeated web_search loops on small reasoning puzzles
- 主要言語
- C
- スター
- 22.3k
- フォーク
- 2.1k
- 平均マージ
- 1日 3時間
- マージ済み PR(30日)
- 4
説明
## Setup
- Build: `54b36ed` (main)
- Model: `DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf` (0731 weight release, IQ2XXS/w2Q2K routed experts)
- Runtime: `ds4-agent` via `./agent.sh`, ctx 131072, Metal backend on an Apple Silicon Mac (M5 Max)
- No DSpark/MTP in use
## Symptom
On small logic-puzzle prompts, the agent burns a large amount of reasoning (~30k tokens) and, based on the reasoning trace, appears to arrive at the correct answer internally — but instead of committing to a final response it repeatedly invokes `web_search_call` to try to verify the answer, and its confidence in its own derived answer never stabilizes.
Two test cases:
1. **Numeric transform puzzle:**
```
11118888888855 -> 118885 | 79999775555 -> 99755 | AAABBBYUDD -> ?
```
Here the run eventually stopped on its own — it found an answer via web search and checked its own derived answer against that, rather than trusting its internal reasoning.
2. **Lateral-thinking riddle:**
```
A feather → Heavy
A ton of bricks → Light
A gold bar → Heavy
A secret → ?
```
This one never stopped by itself — had to be manually interrupted. The model appeared to keep flip-flopping between candidate answers, unable to settle on which one was correct.
I quantized Qwen3.6-35B independently a while back and saw a similar pattern there too (excessive deliberation without commitment on small puzzle-style prompts), so my guess is this points at a routing failure in the MoE experts rather than something specific to this model or quant — possibly aggravated by the low-bit (IQ2XXS/w2Q2K) routed-expert quantization here. Not sure if this is fixable or just an inherent cost of this quant tier, but wanted to flag it in case it's useful data.
コントリビューションガイド
評価
この issue はまだ評価されていません。