MTP draft acceptance collapses to ~0% when VLLM_XPU_ENABLE_XPU_GRAPH=1 (Qwen3.8-27B-FP8, TP=2, 0.21.0-b2)

## Title

MTP draft acceptance collapses to ~0% when `VLLM_XPU_ENABLE_XPU_GRAPH=1` (Qwen3.8-27B-FP8, TP=2, 0.21.0-b2)

---

## Summary

With MTP speculative decoding enabled on Qwen3.8-27B-FP8 (TP=2, 2x Arc Pro B70), the draft head works correctly in eager mode and with torch.compile (64–86% acceptance), but when XPU graph capture is enabled the acceptance rate drops to 0.16% and acceptance length becomes exactly 1.00. Every draft token is rejected on every step. The target model still produces coherent output, so it appears the draft head is being replayed with stale or incorrect inputs under graph capture rather than the target being corrupted.

XPU graph itself works well here — it halves the per-step latency (116ms → 53ms). With a working drafter that would be ~60+ tok/s single-stream, so this is a large performance win sitting behind one bug.

## Environment

- Image: `intel/llm-scaler-vllm:0.21.0-b2`
- vLLM: `0.21.1.dev0+gad7125a43.d20260802` (from startup banner)
- GPUs: 2x Intel Arc Pro B70 (32GB), PCIe Gen3 x8/x8
- Host: Ubuntu 24.04 LTS, kernel 6.17 HWE, Docker
- Model: `Qwen/Qwen3.8-27B-FP8` (pre-quantized, local path), resolves to `Qwen3_5ForConditionalGeneration`, draft resolves to `Qwen3_5MTP`

Note: Qwen3.8-27B is not yet in the supported-models list, but it uses the Qwen3.5 architecture and loads cleanly via the existing Qwen3.5/3.6-27B path.

## Launch command

Identical across all three runs except for the eager flag and the graph env var, as noted per run.

```bash
export ZE_AFFINITY_MASK=0,1
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export CCL_WORKER_COUNT=2
export CCL_TOPO_P2P_ACCESS=0
export FI_PROVIDER=shm
export CCL_ATL_TRANSPORT=ofi
export CCL_ZE_IPC_EXCHANGE=sockets
export CCL_ATL_SHM=1
# Run C only: export VLLM_XPU_ENABLE_XPU_GRAPH=1

vllm serve /llm/models/Qwen3.8-27B-FP8 \
--served-model-name qwen3.8-27b \
--tensor-parallel-size 2 \
--distributed-executor-backend mp \
--dtype float16 \
--trust-remote-code \
--gpu-memory-utilization 0.85 \
--max-model-len 131072 \
--max-num-batched-tokens 16384 \
--enable-chunked-prefill \
--block-size 64 \
--enable-prefix-caching \
--max-num-seqs 16 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
--limit-mm-per-prompt '{"image":0,"video":0}' \
--default-chat-template-kwargs '{"reasoning_effort": "low"}' \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--host 0.0.0.0 --port 8000
# Run A additionally passes --enforce-eager
```

## Benchmark command

```bash
vllm bench serve --model /llm/models/Qwen3.8-27B-FP8 \
--served-model-name qwen3.8-27b --port 8000 --backend vllm \
--dataset-name random --num-prompts 8 --random-input-len 2048 \
--random-output-len 1024 --random-range-ratio 0.5 \
--request-rate inf --max-concurrency 1 \
--percentile-metrics ttft,tpot,itl,e2el \
--trust-remote-code --ignore-eos
```

(Run A also passed `--seed 1337`, which triggers the `topk_topp_sampler ... Falling back to PyTorch-native` path. Runs B and C did not use a seed.)

## Results (single stream, c=1)

| Run | Mode | Mean ITL | Output tok/s | Acceptance rate | Acceptance length | Pos 0 / 1 / 2 |
|---|---|---|---|---|---|---|
| A | `--enforce-eager` | 116.7 ms | 22.6 | 63.8% | 2.91 | 67 / 62 / 62 |
| B | torch.compile, no XPU graph | 105.5 ms | 31.4 | 85.9% | 3.58 | 91 / 86 / 80 |
| C | torch.compile + `VLLM_XPU_ENABLE_XPU_GRAPH=1` | 53.6 ms | 16.9 | **0.16%** | **1.00** | **0.5 / 0 / 0** |

Run C speculative decoding block in full:

```
---------------Speculative Decoding---------------
Acceptance rate (%): 0.16
Acceptance length: 1.00
Drafts: 6618
Draft tokens: 19854
Accepted tokens: 31
Per-position acceptance (%):
Position 0: 0.47
Position 1: 0.00
Position 2: 0.00
```

Run B for comparison:

```
---------------Speculative Decoding---------------
Acceptance rate (%): 85.92
Acceptance length: 3.58
Drafts: 2680
Draft tokens: 8040
Accepted tokens: 6908
Per-position acceptance (%):
Position 0: 91.16
Position 1: 86.38
Position 2: 80.22
```

## Relevant startup log lines (Run C)

```
[gpu_model_runner.py:4882] Loading drafter model...
[llm_base_proposer.py:1290] Detected MTP model. Sharing target model embedding weights with the draft model.
[llm_base_proposer.py:1346] Detected MTP model. Sharing target model lm_head weights with the draft model.
[backends.py:1089] Using cache directory: /root/.cache/vllm/torch_compile_cache/b01531ce11/rank_0_0/backbone for vLLM's torch.compile
[backends.py:1089] Using cache directory: /root/.cache/vllm/torch_compile_cache/b01531ce11/rank_0_0/eagle_head for vLLM's torch.compile
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 19/19 [00:29<00:00, 1.56s/it]
[gpu_model_runner.py:6244] Graph capturing finished in 30 secs, took 4.73 GiB
```

## Observations

- Position-0 acceptance of 0.47% is far below what random guesses from a top-k=20 distribution would produce, so the drafter is emitting something unrelated to the target's next-token distribution, not merely drafting poorly.
- Output text from the target remains coherent in Run C (acceptance length 1.00 means every token is still produced by the target via the bonus token).
- Per-step latency in Run C (53ms) still includes the three wasted draft passes, so the graph-mode step itself is fast; only the draft result is wrong.
- Prefill TTFT is similar across B and C (~2–3s median), consistent with the issue being in the decode/draft path.

## Expected

Acceptance rate under XPU graph comparable to the non-graph run (~85%), yielding roughly 3.5 tokens per 53ms step.

## Happy to provide

Full logs for all three runs, output samples, or re-runs with other flags (e.g. different `cudagraph_mode` values, `num_speculative_tokens=1`) if useful.