Arc Pro B50 (BMG-G21) — torch.compile produces wrong output / SYCL JIT crash on vLLM
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
B50/BMG-G21 - vLLM torch.compile produces semantically wrong output (TP=2) or SYCL JIT crash (TP=1) — workaround: --enforce-eager`
I noticed the repo homepage only describes support for B60s but I thought i'd give this dual b50 setup a shot anyways. albeit w/o XeLink.
On Intel Arc Pro **B50**, `intel/llm-scaler-vllm:0.14.0-b8.2` (and `0.14.0-b8.1`) running Qwen3 family models with `torch.compile` enabled (default mode) either:
- **Crashes JIT** with `terminate called after throwing an instance of 'sycl::_V1::exception'` on `--tensor-parallel-size 1`, **OR**
- **Produces semantically wrong output** (coherent-looking but prompt-disconnected token loops) on `--tensor-parallel-size 2`.
Adding `--enforce-eager` works around it (correct output, ~25 tok/s on Qwen3-8B BF16 TP=2).
This appear related to closed issue [intel/intel-xpu-backend-for-triton#5581](https://github.com/intel/intel-xpu-backend-for-triton/issues/5581) ("Incorrect compiler behavior with vllm tiling"), but on a SKU (B50) that wasn't part of that fix validation matrix.
## Environment
- **Hardware:** 2× Intel Arc Pro B50 (BMG-G21, 16 Xe-cores, 16 GB VRAM each, ReBAR enabled, both BARs = 16 GB)
- **Host:** Ubuntu 24.04, kernel `6.17.0-22-generic`, `xe` driver
- **Image:** `intel/llm-scaler-vllm:0.14.0-b8.2` (also reproduced on `0.14.0-b8.1`)
- **vLLM:** `0.14.1.dev0+gb17039bcc.d20260311`
- **Compute runtime (inside container):** NEO `25.48.36300.8-1~24.04~ppa3` (= **Agama 1208**), Level Zero loader `1.26.2`, IGC bundled
- **PyTorch:** `2.10.0+xpu`
- **`sycl-ls` confirms:** `[level_zero:gpu] Intel(R) Arc(TM) Pro B50 Graphics 20.1.0 [1.14.36300+8]` (×2)
- **`torch.xpu`:** detects both cards correctly, 15.13 GiB each
## Reproduction
```bash
docker run --rm \
--device /dev/dri:/dev/dri \
--group-add 44 --group-add 993 \
--shm-size 16g --ipc host \
-v /tmp/hf_cache:/root/.cache/huggingface \
-p 8000:8000 \
--entrypoint vllm \
intel/llm-scaler-vllm:0.14.0-b8.2 \
serve Qwen/Qwen3-8B \
--tensor-parallel-size 2 \
--dtype bfloat16 \
--host 0.0.0.0 --port 8000 \
--gpu-memory-utilization 0.6 \
--max-model-len 4096
```
Then:
```bash
curl http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "Qwen/Qwen3-8B",
"messages": [{"role":"user","content":"What is 2+2?"}],
"max_tokens": 100,
"temperature": 0,
"chat_template_kwargs": {"enable_thinking": false}
}'
```
Contributor guide
Assessment
This issue has not been assessed yet.