intel / intel/llm-scaler

Arc Pro B50 (BMG-G21) — torch.compile produces wrong output / SYCL JIT crash on vLLM

Open
#392 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

B50/BMG-G21 - vLLM torch.compile produces semantically wrong output (TP=2) or SYCL JIT crash (TP=1) — workaround: --enforce-eager`

I noticed the repo homepage only describes support for B60s but I thought i'd give this dual b50 setup a shot anyways. albeit w/o XeLink.

On Intel Arc Pro **B50**, `intel/llm-scaler-vllm:0.14.0-b8.2` (and `0.14.0-b8.1`) running Qwen3 family models with `torch.compile` enabled (default mode) either:

- **Crashes JIT** with `terminate called after throwing an instance of 'sycl::_V1::exception'` on `--tensor-parallel-size 1`, **OR**
- **Produces semantically wrong output** (coherent-looking but prompt-disconnected token loops) on `--tensor-parallel-size 2`.

Adding `--enforce-eager` works around it (correct output, ~25 tok/s on Qwen3-8B BF16 TP=2).

This appear related to closed issue [intel/intel-xpu-backend-for-triton#5581](https://github.com/intel/intel-xpu-backend-for-triton/issues/5581) ("Incorrect compiler behavior with vllm tiling"), but on a SKU (B50) that wasn't part of that fix validation matrix.

## Environment

- **Hardware:** 2× Intel Arc Pro B50 (BMG-G21, 16 Xe-cores, 16 GB VRAM each, ReBAR enabled, both BARs = 16 GB)
- **Host:** Ubuntu 24.04, kernel `6.17.0-22-generic`, `xe` driver
- **Image:** `intel/llm-scaler-vllm:0.14.0-b8.2` (also reproduced on `0.14.0-b8.1`)
- **vLLM:** `0.14.1.dev0+gb17039bcc.d20260311`
- **Compute runtime (inside container):** NEO `25.48.36300.8-1~24.04~ppa3` (= **Agama 1208**), Level Zero loader `1.26.2`, IGC bundled
- **PyTorch:** `2.10.0+xpu`
- **`sycl-ls` confirms:** `[level_zero:gpu] Intel(R) Arc(TM) Pro B50 Graphics 20.1.0 [1.14.36300+8]` (×2)
- **`torch.xpu`:** detects both cards correctly, 15.13 GiB each

## Reproduction

```bash
docker run --rm \
--device /dev/dri:/dev/dri \
--group-add 44 --group-add 993 \
--shm-size 16g --ipc host \
-v /tmp/hf_cache:/root/.cache/huggingface \
-p 8000:8000 \
--entrypoint vllm \
intel/llm-scaler-vllm:0.14.0-b8.2 \
serve Qwen/Qwen3-8B \
--tensor-parallel-size 2 \
--dtype bfloat16 \
--host 0.0.0.0 --port 8000 \
--gpu-memory-utilization 0.6 \
--max-model-len 4096
```

Then:

```bash
curl http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "Qwen/Qwen3-8B",
"messages": [{"role":"user","content":"What is 2+2?"}],
"max_tokens": 100,
"temperature": 0,
"chat_template_kwargs": {"enable_thinking": false}
}'
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.