DSpark: per-accepted-token replay cancels ~100% of the speculative saving (M3 Ultra measurements, net_saved = -3.98 s)

## Summary

On a Mac Studio M3 Ultra, DSpark is a consistent **~35% throughput loss**, not a gain. `DS4_DSPARK_STATS=1` shows why: the rollback+replay path introduced by af80694 and 7fb2830 costs almost exactly what the accepted drafts saved, so the theoretical ceiling for net gain is ~0 — and then propose + verify are charged on top.

I understand from the comment in af80694 that this is a known and deliberate trade ("*this makes DSpark's already-negative throughput case worse, but correctness comes first*"). This report is not a request to revert it. It is an attempt to **quantify** how large that cost is on real hardware, since the comment only describes it qualitatively, and to point out that the replay cost appears to scale with the acceptance rate — meaning the better the drafter performs, the more the replay takes back.

## Environment

| | |
|---|---|
| Machine | Mac Studio M3 Ultra, 80-core GPU, 512 GB unified memory |
| Backend | Metal. `Metal 4 tensor API disabled for pre-M5/pre-A19 devices` |
| ds4 | `b030961` (2026-08-05), `make` |
| Model | `DeepSeek-V4-Flash-Q4KExperts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-imatrix-0731.gguf` |
| DSpark | `DeepSeek-V4-Flash-DSpark-support-0731.gguf` — `stages=3 block=5 markov_rank=256 tensors=81 missing=0 invalid=0` |

## Measured throughput

Same prompt, same binary, `--temp 0 --nothink -n 256`, only `--mtp ... --dspark` added:

| Workload | no DSpark | with DSpark | ratio |
|---|---:|---:|---:|
| Italian prose continuation (6244-token prompt) | 33.54 t/s | 21.77 t/s | **0.649x** |
| C code generation (26-token prompt) | 36.50 t/s | 23.37 t/s | **0.640x** |

Prefill was unchanged (554.26 → 553.31 t/s), as documented. Generated text was identical in both runs, so acceptance/verification is behaving correctly.

The two ratios being nearly identical across very different content is itself informative: this is not acceptance-rate variance, it is a fixed structural overhead.

## `DS4_DSPARK_STATS=1` breakdown

From the C code-generation run:

```
ds4: DSpark stats cycles=126 first_tokens=126 proposed=203 accepted_draft=121
accept_rate=59.61% avg_accept=0.960 full=27 partial=22 miss_first=11 no_draft=66
no_room=0 invalid=0 scheduler_skips=54 tail_skips=8 verifier_unavailable=0 errors=0
time_ms propose=588.934 prop_stage0=0.003 prop_setup=34.369 prop_cache=3.686
prop_chain=369.871 prop_hidden=0.003 prop_conf0=0.910 prop_logits=124.131
prop_markov=55.702 prop_confidence=0.000 snapshot=25.165 verify=3276.689
verify_upload=1.210 verify_layer=3275.356 verify_head=0.000 verify_read=0.090
verify_fused_head=49 replay=3336.601 spec_total=6668.069 target=3683.981
saved=3278.481 net_saved=-3978.522
draft_len_hist=1:6,2:11,3:12,4:16,5:15
accepted_len_hist=0:77,1:14,2:15,3:9,4:5,5:6
```

## Analysis

**The drafter itself is working well.** `accept_rate=59.61%`, `avg_accept=0.960` — nearly one extra token per cycle. That is a healthy acceptance rate that should translate into a large win.

**The replay takes all of it back:**

```
saved = 3278.481 ms (121 accepted tokens x ~27.1 ms target eval)
replay = 3336.601 ms (121 accepted tokens x ~27.6 ms ordinary decode)
--------------
net from acceptance = -58 ms
```

`replay / saved = 1.018`. Per-token, replay costs the same as the ordinary decode step it replaces, which is expected — it *is* the same kernel (`metal_graph_eval_token_raw_swa`), called once per accepted draft token in the loop at `ds4.c`.

Everything else is then pure additional cost:

```
propose 588.9 ms
verify 3276.7 ms (verify_layer 3275.4)
snapshot 25.2 ms
--------
3890.8 ms -> net_saved = -3978.5 ms
```

For reference, a single target token eval here is `target/first_tokens = 3683.981/126 = 29.2 ms`, while one verify call is `verify_layer/49 = 66.8 ms` — about 2.3x a single-token forward for a 5-token block.

The scheduler is already trying to limit the damage: `no_draft=66` with `scheduler_skips=54` means it declined to draft in 43% of cycles. Even so the run was 36% slower.

## Why this looks device-independent

Two of the three costs are plausibly worse on my machine specifically:

- `verify_layer` at 2.3x a single-token forward may be cheaper on M5-class hardware, where the Metal 4 tensor API paths are available (my log line: `Metal 4 tensor API disabled for pre-M5/pre-A19 devices`).
- `propose` likewise.

But **replay is not**: it invokes the same single-token decode kernel that ordinary generation uses, so on any backend `replay ≈ saved` by construction. Even if propose and verify were free, this run's net would be roughly `-58 ms`, i.e. break-even at best. That is what makes me think the ceiling is structural rather than a tuning problem.

If that reading is wrong and there is a configuration in which DSpark is currently a net win, I would be glad to be corrected — and happy to run whatever measurement would be useful.

## Reproduction

```sh
./download_model.sh ds4f-q4
./download_model.sh ds4f-dspark

# baseline
./ds4 -m ./ds4flash.gguf --nothink --temp 0 -n 256 \
-p "Write a C function that reverses a singly linked list, with full edge-case handling and comments."

# with DSpark + stats
DS4_DSPARK_STATS=1 ./ds4 -m ./ds4flash.gguf --nothink --temp 0 -n 256 \
--mtp gguf/DeepSeek-V4-Flash-DSpark-support-0731.gguf --dspark \
-p "Write a C function that reverses a singly linked list, with full edge-case handling and comments."
```

## Questions

Since the root cause is that the batched verify GEMM (`ds4_gpu_matmul_f16_pair_tensor`) and the fused single-token projection+store (`ds4_gpu_matmul_f16_pair_compressor_store_tensor`) produce numerically distinct compressor-frontier state, is there a path that avoids the per-token replay? A few directions, offered only as questions since I do not know the constraints:

1. Could the verify batch write its compressor frontier through the same fused kernel ordinary decode uses, so the states are bit-identical and the accepted prefix can be committed directly?
2. Alternatively, could an opt-in flag (say `--dspark-fast-accept`) restore the pre-af80694 commit path for users who accept that greedy output may diverge from non-speculative decode? The README already describes DSpark as experimental and opt-in.
3. Would it be worth having the scheduler measure `net_saved` over a rolling window and auto-disable drafting when it stays negative, so the current behaviour degrades to baseline speed instead of 0.64x?

Happy to test patches on this hardware.