[Regression 0.21.0-b1] Qwen3.6-27B gdn_attention XPU intermittently corrupts prefill; first token becomes "!" with identical greedy input
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
## Summary
Qwen3.6-27B on llm-scaler-vllm:0.21.0-b1 intermittently produces
`!!!!!!!!` from the first generated token.
This appears to be a regression or incomplete backport of #483, which was
reported as fixed in 0.14.0-b8.3.2.
The issue is independent of OpenCode and is reproducible through
`/v1/completions` using captured prompt token IDs.
## Controlled reproduction
Model:
`Qwen3.6-27B-int4-AutoRound`
Hardware:
- Intel Arc Pro B70 32 GB
- Single GPU
- Ubuntu 26.04
- Ryzen 7 9700X
- 64 GB RAM
Server:
- llm-scaler-vllm:0.21.0-b1
- dtype=float16
- kv-cache-dtype=fp8
- max-num-seqs=1
- max-num-batched-tokens=4096
- no async scheduling
- no prefix caching during the controlled test
- Mamba cache dtype left at auto
- XPU Graph disabled during the controlled test
Captured prompt:
- 111,667 exact token IDs
- temperature=0
- max_tokens=1 or streaming
- identical input on every execution
Observed result:
Attempts using the exact same token IDs:
1. correct response
2. correct response
3. correct response
4. first generated characters are `!!!!!!!!`
The failure starts at output position zero, so corruption is already present
after prefill, before decode accumulation.
A separate controlled run with 113,495 exact token IDs produced failures at
iterations 2, 3, 5, 8 and 11 in different test configurations.
## Evidence of non-finite values
Requesting logprobs eventually caused:
`ValueError: Out of range float values are not JSON compliant: nan`
This shows that the corrupted execution can produce NaN logprob values.
Inductor `nan_asserts=true` also raised, although one assertion was triggered
on `z` before the mutating GDN custom op filled that `empty_like` buffer, so
that particular assertion was premature and is not by itself proof of the
root tensor.
## Suspected kernel
The active fallback path calls:
`torch.ops.vllm.gdn_attention_core_xpu(...)`
which dispatches to:
`torch.ops._xpu_C.gdn_attention(...)`
in:
`vllm/_xpu_ops.py::_gdn_attention_core_xpu_impl`
The 0.21.0-b1 source itself contains this comment near the call site:
`the GDN core is the !!!! OOB-corruption region`
Prefill and mixed batches continue using the upstream SYCL GDN kernel.
## Eliminated conditions
The issue still occurs with:
- temperature=0
- max_num_seqs=1
- async scheduling disabled
- prefix caching disabled
- XPU Graph disabled
- packed recurrent decode disabled
- all DISABLE_ESIMD_GDN_* variables enabled
- RMSNormGated XPU kernel replaced temporarily by forward_native
- causal-conv metadata copies made synchronous
- Mamba SSM cache in float32
- both AutoRound checkpoint and online sym_int4 observations
Changing `z = torch.empty_like(core_attn_out)` to `zeros_like` passed a short
20-run test but still failed under real workload, so it is not a complete fix.
## Additional dtype bug
`--mamba-cache-dtype float32` causes deterministic `!!!!!!!!` even for a valid
8,420-token prompt, while Mamba cache dtype `auto` produces coherent output
with the same exact tokens.
## Reproducer artifacts available
- Exact prompt-token JSON for a 111,667-token failure
- Exact prompt-token JSON for a 113,495-token failure
- Streaming reproducer that aborts after 8 consecutive exclamation marks
- Repeat-run JSON results
- Full DEBUG vLLM logs
- Generated Inductor graph and traceback
Contributor guide
Assessment
This issue has not been assessed yet.