antirez / antirez/ds4

DeepSeek-V4-Flash: native MTP --mtp-draft>=2 corrupts output (BOS-token injection) at default (non-strict) verify — root cause + fix

Đang mở
#750 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

### Summary

With native MTP speculative decoding at `--mtp-draft >= 2` (server defaults, i.e. no `--quality`), DeepSeek-V4-Flash output is corrupted by injected BOS tokens — e.g. `We need<|begin▁of▁sentence|># The sky is<|begin▁of▁sentence|># ...`. `--mtp-draft 1` is clean.

### Root cause (verified)

In `ds4.c` `ds4_session_eval_speculative_argmax`: without `--quality`, `strict_mtp` is false, so `use_decode2_exact` is false and draft>=2 uses the **non-exact** batched verifier `metal_graph_verify_suffix_tops` (`snapshot_required=false`) plus the no-snapshot partial-accept rollback `spec_frontier_commit_prefix1`. On a 1-of-2 partial accept, `s->logits` is set to a corrupt row-0 whose argmax is token id 0 (`<|begin▁of▁sentence|>`), so the next emitted token is BOS → self-restart cascade. Confirmed with `DS4_DSPARK_STATS`/`DS4_MTP_CONF_LOG` (`drafted=2 committed=1 target_next=0`). A correct verifier would only *reject* the bad drafts; the corruption requires the non-exact verify/rollback to wrongly *accept*.

### Fix

Route draft>=2 through the exact verifier by default (`decode2_exact` for draft==2; exact sequential fallback for draft>2 — the same path that is clean at draft=1). Verified: patched draft-1/2/4 outputs are **bit-identical** and coherent. (No throughput change — DS4 MTP gives no single-token speedup on ds4-server anyway; this is purely a correctness fix.)

### Minor

`DS4_DSPARK_STATS=1` prints per-cycle accept_rate / avg_accept / net_saved (flushed at exit or on SIGINT, which doesn't kill the server) — very useful and currently undocumented.

Patch available. Tested on GB10 (sm_121a), DeepSeek-V4-Flash-IQ2XXS chat-v2 GGUF + the MTP-Q4K drafter.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.