RuntimeError: query and key must have the same dtype with intel/llm-scaler-vllm:0.14.0-b8.3

# docker cmd

```
docker run -td \
--privileged \
--net=host \
--device=/dev/dri \
--name=lsv-container \
-v /root/data/ai/model:/llm/models/ \
-e no_proxy=localhost,127.0.0.1 \
-e http_proxy=$http_proxy \
-e https_proxy=$https_proxy \
--shm-size="480g" \
--entrypoint /bin/bash \
intel/llm-scaler-vllm:0.14.0-b8.3
```

# python cmd

```
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
python3 -m vllm.entrypoints.openai.api_server \
--model /llm/models/huggingface/Qwen/Qwen3-Coder-30B-A3B-Instruct/ \
--served-model-name Qwen3-Coder-30B-A3B-Instruct \
--api-key ********** \
--dtype=auto\
--enforce-eager \
--port 8000 \
--host 0.0.0.0 \
--trust-remote-code \
--disable-sliding-window \
--gpu-memory-util=0.9 \
--no-enable-prefix-caching \
--max-num-batched-tokens=65536 \
--disable-log-requests \
--max-model-len=65536 \
--block-size 64 \
--quantization fp8 \
--enable-expert-parallel \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
-tp=4
```

# when occur

after the service established, a simple POST /v1/chat/completions caused crash.

```
raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 504 - {'error': {'message': ' (request id: 2026060115094951576597213269479)', 'type': 'upstream_error', 'param': '504', 'code': 'bad_response_status_code'}}
```

# error logs

```
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] WorkerProc hit an exception.
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] Traceback (most recent call last):
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 817, in worker_busy_loop
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] output = func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/worker_base.py", line 365, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.worker.execute_model(scheduler_output)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 645, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] output = self.model_runner.execute_model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3472, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] model_output = self._model_forward(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3091, in _model_forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 833, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states = self.model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/compilation/decorators.py", line 390, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.forward(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 458, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states, residual = layer(positions, hidden_states, residual)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 381, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states = self.self_attn(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 312, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] attn_output = self.attn(q, k, v)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 388, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] torch.ops.vllm.unified_attention_with_output(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1209, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._op(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/utils/kv_transfer_utils.py", line 39, in wrapper
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 878, in unified_attention_with_output
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] self.impl.forward(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/flash_attn.py", line 774, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] flash_attn_varlen_func(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/fa_utils.py", line 70, in flash_attn_varlen_func
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] result = _cutlass_flash_attn_varlen_func(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm_xpu_kernels/flash_attn_interface.py", line 125, in flash_attn_varlen_func
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] out, softmax_lse = torch.ops._vllm_fa2_C.varlen_fwd(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1209, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._op(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] RuntimeError: query and key must have the same dtype
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] Traceback (most recent call last):
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 817, in worker_busy_loop
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] output = func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/worker_base.py", line 365, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.worker.execute_model(scheduler_output)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 645, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] output = self.model_runner.execute_model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3472, in execute_model
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] model_output = self._model_forward(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 3091, in _model_forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 833, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states = self.model(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/compilation/decorators.py", line 390, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self.forward(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 458, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states, residual = layer(positions, hidden_states, residual)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 381, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] hidden_states = self.self_attn(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 312, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] attn_output = self.attn(q, k, v)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._call_impl(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return forward_call(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 388, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] torch.ops.vllm.unified_attention_with_output(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1209, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._op(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/utils/kv_transfer_utils.py", line 39, in wrapper
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return func(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 878, in unified_attention_with_output
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] self.impl.forward(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/flash_attn.py", line 774, in forward
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] flash_attn_varlen_func(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/attention/backends/fa_utils.py", line 70, in flash_attn_varlen_func
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] result = _cutlass_flash_attn_varlen_func(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/vllm_xpu_kernels/flash_attn_interface.py", line 125, in flash_attn_varlen_func
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] out, softmax_lse = torch.ops._vllm_fa2_C.varlen_fwd(
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1209, in __call__
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] return self._op(*args, **kwargs)
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] ^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822] RuntimeError: query and key must have the same dtype
(Worker_TP0_EP0 pid=4209) ERROR 06-01 07:09:47 [multiproc_executor.py:822]
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [dump_input.py:72] Dumping input data for V1 LLM engine (v0.14.1.dev0+gb17039bcc.d20260527) with config: model='/llm/models/huggingface/Qwen/Qwen3-Coder-30B-A3B-Instruct/', speculative_config=None, tokenizer='/llm/models/huggingface/Qwen/Qwen3-Coder-30B-A3B-Instruct/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=65536, download_dir=None, load_format=auto, tensor_parallel_size=4, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=fp8, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=xpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=Qwen3-Coder-30B-A3B-Instruct, enable_prefix_caching=False, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': , 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'splitting_ops': [], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [65536], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': , 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': None, 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': None, 'dynamic_shapes_config': {'type': , 'evaluate_guards': False, 'assume_32_bit_indexing': True}, 'local_cache_dir': None},
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [dump_input.py:79] Dumping scheduler output for model execution: SchedulerOutput(scheduled_new_reqs=[], scheduled_cached_reqs=CachedRequestData(req_ids=['chatcmpl-8430a412986558ce-b5b28daa'],resumed_req_ids=set(),new_token_ids_lens=[],all_token_ids_lens={},new_block_ids=[None],num_computed_tokens=[49],num_output_tokens=[1]), num_scheduled_tokens={chatcmpl-8430a412986558ce-b5b28daa: 1}, total_num_scheduled_tokens=1, scheduled_spec_decode_tokens={}, scheduled_encoder_inputs={}, num_common_prefix_blocks=[0], finished_req_ids=[], free_encoder_mm_hashes=[], preempted_req_ids=[], has_structured_output_requests=false, pending_structured_output_tokens=false, num_invalid_spec_tokens=null, kv_connector_metadata=null, ec_connector_metadata=null, new_block_ids_to_zero=null)
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [dump_input.py:81] Dumping scheduler stats: SchedulerStats(num_running_reqs=1, num_waiting_reqs=0, step_counter=0, current_wave=0, kv_cache_usage=0.00017421602787459634, prefix_cache_stats=PrefixCacheStats(reset=False, requests=0, queries=0, hits=0, preempted_requests=0, preempted_queries=0, preempted_hits=0), connector_prefix_cache_stats=None, kv_cache_eviction_events=[], spec_decoding_stats=None, kv_connector_stats=None, waiting_lora_adapters={}, running_lora_adapters={}, cudagraph_stats=None, perf_stats=None)
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] EngineCore encountered a fatal error.
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] Traceback (most recent call last):
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 929, in run_engine_core
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] engine_core.run_busy_loop()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 956, in run_busy_loop
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] self._process_engine_step()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 989, in _process_engine_step
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] outputs, model_executed = self.step_fn()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] ^^^^^^^^^^^^^^
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 388, in step
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] model_output = future.result()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] ^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 80, in result
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] return super().result()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] ^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] return self.__get_result()
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] ^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] raise self._exception
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 84, in wait_for_response
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] response = self.aggregate(get_response())
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] ^^^^^^^^^^^^^^
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 342, in get_response
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] raise RuntimeError(
(EngineCore_DP0 pid=3947) ERROR 06-01 07:09:47 [core.py:938] RuntimeError: Worker failed with error 'query and key must have the same dtype', please check the stack trace above for the root cause
(Worker_TP3_EP3 pid=4212) INFO 06-01 07:09:47 [multiproc_executor.py:707] Parent process exited, terminating worker
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] AsyncLLM output_handler failed.
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] Traceback (most recent call last):
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 502, in output_handler
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] outputs = await engine_core.get_output_async()
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 899, in get_output_async
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] raise self._format_exception(outputs) from None
(APIServer pid=3681) ERROR 06-01 07:09:47 [async_llm.py:546] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(Worker_TP2_EP2 pid=4211) INFO 06-01 07:09:47 [multiproc_executor.py:707] Parent process exited, terminating worker
(Worker_TP0_EP0 pid=4209) INFO 06-01 07:09:47 [multiproc_executor.py:707] Parent process exited, terminating worker
(Worker_TP1_EP1 pid=4210) INFO 06-01 07:09:47 [multiproc_executor.py:707] Parent process exited, terminating worker
```