antirez / antirez/ds4

DeepSeek-V4-Flash: native MTP --mtp-draft>=2 corrupts output (BOS-token injection) at default (non-strict) verify — root cause + fix

Abierto
#750 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

### Summary

With native MTP speculative decoding at `--mtp-draft >= 2` (server defaults, i.e. no `--quality`), DeepSeek-V4-Flash output is corrupted by injected BOS tokens — e.g. `We need<|begin▁of▁sentence|># The sky is<|begin▁of▁sentence|># ...`. `--mtp-draft 1` is clean.

### Root cause (verified)

In `ds4.c` `ds4_session_eval_speculative_argmax`: without `--quality`, `strict_mtp` is false, so `use_decode2_exact` is false and draft>=2 uses the **non-exact** batched verifier `metal_graph_verify_suffix_tops` (`snapshot_required=false`) plus the no-snapshot partial-accept rollback `spec_frontier_commit_prefix1`. On a 1-of-2 partial accept, `s->logits` is set to a corrupt row-0 whose argmax is token id 0 (`<|begin▁of▁sentence|>`), so the next emitted token is BOS → self-restart cascade. Confirmed with `DS4_DSPARK_STATS`/`DS4_MTP_CONF_LOG` (`drafted=2 committed=1 target_next=0`). A correct verifier would only *reject* the bad drafts; the corruption requires the non-exact verify/rollback to wrongly *accept*.

### Fix

Route draft>=2 through the exact verifier by default (`decode2_exact` for draft==2; exact sequential fallback for draft>2 — the same path that is clean at draft=1). Verified: patched draft-1/2/4 outputs are **bit-identical** and coherent. (No throughput change — DS4 MTP gives no single-token speedup on ds4-server anyway; this is purely a correctness fix.)

### Minor

`DS4_DSPARK_STATS=1` prints per-cycle accept_rate / avg_accept / net_saved (flushed at exit or on SIGINT, which doesn't kill the server) — very useful and currently undocumented.

Patch available. Tested on GB10 (sm_121a), DeepSeek-V4-Flash-IQ2XXS chat-v2 GGUF + the MTP-Q4K drafter.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.