antirez / antirez/ds4

Model reaches correct answer internally but won't commit — repeated web_search loops on small reasoning puzzles

Đang mở
#646 3 bình luận 1 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

## Setup

- Build: `54b36ed` (main)
- Model: `DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf` (0731 weight release, IQ2XXS/w2Q2K routed experts)
- Runtime: `ds4-agent` via `./agent.sh`, ctx 131072, Metal backend on an Apple Silicon Mac (M5 Max)
- No DSpark/MTP in use

## Symptom

On small logic-puzzle prompts, the agent burns a large amount of reasoning (~30k tokens) and, based on the reasoning trace, appears to arrive at the correct answer internally — but instead of committing to a final response it repeatedly invokes `web_search_call` to try to verify the answer, and its confidence in its own derived answer never stabilizes.

Two test cases:

1. **Numeric transform puzzle:**
```
11118888888855 -> 118885 | 79999775555 -> 99755 | AAABBBYUDD -> ?
```
Here the run eventually stopped on its own — it found an answer via web search and checked its own derived answer against that, rather than trusting its internal reasoning.

2. **Lateral-thinking riddle:**
```
A feather → Heavy
A ton of bricks → Light
A gold bar → Heavy
A secret → ?
```
This one never stopped by itself — had to be manually interrupted. The model appeared to keep flip-flopping between candidate answers, unable to settle on which one was correct.

I quantized Qwen3.6-35B independently a while back and saw a similar pattern there too (excessive deliberation without commitment on small puzzle-style prompts), so my guess is this points at a routing failure in the MoE experts rather than something specific to this model or quant — possibly aggravated by the low-bit (IQ2XXS/w2Q2K) routed-expert quantization here. Not sure if this is fixable or just an inherent cost of this quant tier, but wanted to flag it in case it's useful data.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.