PyTorch 2.x runtime error
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
Docker image version: 0.14.0-b8.3
GPU: Intel Arc B70
Model: Qwen3.5-9B
Dtype: BFloat16
Runtime Error: (EngineCore_DP0 pid=181) ERROR 06-14 07:26:10 [core.py:936] RuntimeError: Inference tensors do not track version counter.
...
(EngineCore_DP0 pid=181) ERROR 06-14 07:26:10 [core.py:936] File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_5.py", line 428, in forward_xpu
(EngineCore_DP0 pid=181) ERROR 06-14 07:26:10 [core.py:936] self.out_proj.weight.data = self.out_proj.weight.data.clone()
My workaround is to add " --enforce-eager".
Full Docker CMD: docker run -td --privileged --net=host --device=/dev/dri --shm-size="32g" --name vllm-arc-b70 \
-v /sys:/sys:ro \
-v /home/zhou80bin/Models/Qwen3.5:/llm/models/ \
-e HF_HUB_OFFLINE=1 \
--pull=never \
--entrypoint "" \
intel/llm-scaler-vllm:0.14.0-b8.3 \
vllm serve /llm/models/ --served-model-name Qwen3.5 --trust-remote-code \
--gpu-memory-utilization 0.85 --max-model-len 8192 --host 0.0.0.0 --port 8000 \
--enforce-eager
Contributor guide
Assessment
This issue has not been assessed yet.