torch._dynamo.exc.Unsupported: _has_xmx crash when running 32B model on single Arc B60 Pro
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
**Image:** intel/llm-scaler-vllm:latest (45.8GB) and intel/llm-scaler-vllm:0.14.0-b8.1 (48.3GB) — both affected
**Hardware:** Intel Arc Pro B60 (single card, 24GB), Ubuntu 24.04 LTS, kernel 6.17, xe driver
**Model:** Qwen/Qwen3-32B-AWQ
**Command:**
docker run --rm \
--device /dev/dri:/dev/dri \
--device /dev/dri/by-path:/dev/dri/by-path \
--group-add [render GID] --group-add [video GID] \
-v /home/todd/models:/models \
-v /home/todd/vllm-cache:/root/.cache/vllm \
-e VLLM_TARGET_DEVICE=xpu \
--entrypoint "" \
intel/llm-scaler-vllm:latest \
vllm serve /models/Qwen3-32B-AWQ --dtype bfloat16
**Error (during compilation phase, after model shards load):**
torch._dynamo.exc.Unsupported: call_function ipex._C._has_xmx in skip_files
File ".../intel_extension_for_pytorch/transformers/optimize.py"
and ipex._C._has_xmx(0)
**Root cause:**
ipex._C._has_xmx is a C++ pybind function. torch.dynamo cannot trace it,
but it is NOT registered in allow_in_graph unlike the other IPEX C++ functions.
In intel_extension_for_pytorch/frontend.py, these are registered:
torch._dynamo.allow_in_graph(_IPEXLinear)
torch._dynamo.allow_in_graph(_IPEXConv1d)
... etc
But ipex._C._has_xmx is NOT registered, so dynamo crashes when it tries
to trace through the quantization path in transformers/optimize.py.
**Fix (one line, in frontend.py):**
torch._dynamo.allow_in_graph(ipex._C._has_xmx)
torch._dynamo.allow_in_graph(ipex._C._has_2d_block_array) # same issue
**Workaround:** --enforce-eager bypasses compilation but gives ~3 t/s on
32B (no XMX engagement). Not a viable solution for 32B inference.
**Note:** Model loads fine, Flash Attention initializes, crash only occurs
during the torch.compile/inductor step. With enforce-eager the model runs
but without XMX kernels (~3 t/s vs expected 20-40 t/s).
Contributor guide
Assessment
This issue has not been assessed yet.