antirez / antirez/ds4

DeepSeek-V4-Flash: native MTP --mtp-draft>=2 corrupts output (BOS-token injection) at default (non-strict) verify — root cause + fix

Aperta
#750 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
22.3k
Fork
2.1k
Merge medio
1g 3h
PR unite (30g)
4

Descrizione

### Summary

With native MTP speculative decoding at `--mtp-draft >= 2` (server defaults, i.e. no `--quality`), DeepSeek-V4-Flash output is corrupted by injected BOS tokens — e.g. `We need<|begin▁of▁sentence|># The sky is<|begin▁of▁sentence|># ...`. `--mtp-draft 1` is clean.

### Root cause (verified)

In `ds4.c` `ds4_session_eval_speculative_argmax`: without `--quality`, `strict_mtp` is false, so `use_decode2_exact` is false and draft>=2 uses the **non-exact** batched verifier `metal_graph_verify_suffix_tops` (`snapshot_required=false`) plus the no-snapshot partial-accept rollback `spec_frontier_commit_prefix1`. On a 1-of-2 partial accept, `s->logits` is set to a corrupt row-0 whose argmax is token id 0 (`<|begin▁of▁sentence|>`), so the next emitted token is BOS → self-restart cascade. Confirmed with `DS4_DSPARK_STATS`/`DS4_MTP_CONF_LOG` (`drafted=2 committed=1 target_next=0`). A correct verifier would only *reject* the bad drafts; the corruption requires the non-exact verify/rollback to wrongly *accept*.

### Fix

Route draft>=2 through the exact verifier by default (`decode2_exact` for draft==2; exact sequential fallback for draft>2 — the same path that is clean at draft=1). Verified: patched draft-1/2/4 outputs are **bit-identical** and coherent. (No throughput change — DS4 MTP gives no single-token speedup on ds4-server anyway; this is purely a correctness fix.)

### Minor

`DS4_DSPARK_STATS=1` prints per-cycle accept_rate / avg_accept / net_saved (flushed at exit or on SIGINT, which doesn't kill the server) — very useful and currently undocumented.

Patch available. Tested on GB10 (sm_121a), DeepSeek-V4-Flash-IQ2XXS chat-v2 GGUF + the MTP-Q4K drafter.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.