intel / intel/llm-scaler

Qwen3_5ForConditionalGeneration fails to load on single Intel Arc Pro B70 (32GB) due to SYCL crash in vllm registry subprocess

Open
#435 10 comments 0 reactions 1 assignee Claimed by @liu-shaojun View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

Environment:

Hardware: Intel Arc Pro B70 (32GB VRAM), Intel Arc B580 (12GB, secondary), Intel iGPU
OS: Unraid 7.x
Container image: intel/llm-scaler-vllm:latest (v0.14.1.dev0+gb17039bcc.d20260527)
Model: Qwen/Qwen3.6-27B-FP8 (30.9GB, architecture: Qwen3_5ForConditionalGeneration)
ONEAPI_DEVICE_SELECTOR=level_zero:0
ZES_ENABLE_SYSMAN=1

Problem:
When serving Qwen/Qwen3.6-27B-FP8, vllm crashes immediately during model architecture inspection with:
terminate called after throwing an instance of 'sycl::_V1::exception'
what(): No device of requested type available.
Followed by:
RuntimeError: Error raised in subprocess:
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'vllm.model_executor.models.registry']' died with .
pydantic_core.ValidationError: Model architectures ['Qwen3_5ForConditionalGeneration'] failed to be inspected.
Root cause:
vllm's model registry inspector spawns a subprocess to inspect the model architecture. This subprocess imports qwen3_5.py which unconditionally imports qwen3_vl.py at the top level (line 115):
pythonfrom .qwen3_vl import (...)
qwen3_vl.py triggers SYCL device initialization at import time. The subprocess spawned by registry.py does NOT inherit the parent process environment including ONEAPI_DEVICE_SELECTOR, so it sees no XPU device and crashes with SIGABRT before any model loading occurs.
Evidence:
The device IS accessible in the main process:
pythonimport torch; print(torch.xpu.device_count()) # returns 1
The subprocess crash happens purely at import time, before any weights are loaded.
Things tried:

Adding env=os.environ to subprocess.run() in registry.py — does not fix it because the SYCL crash happens during Python import, not device selection
Setting language_model_only=True in config.json — does not prevent the vision import in the subprocess
Adding ONEAPI_DEVICE_SELECTOR=level_zero:0 to /etc/environment — does not fix it
Wrapping the from .qwen3_vl import in try/except in qwen3_5.py — partially fixes the import but then fails at actual model loading

Expected behavior:
Qwen3_5ForConditionalGeneration with language_model_only=True should load and serve correctly on a single B70 with 32GB VRAM. The model is 30.9GB FP8, which fits comfortably.
Note:
This is related to issue #423. The same image successfully serves Qwen3ForCausalLM models (e.g. Qwen/Qwen3-0.6B) without issues. The problem is specific to the Qwen3_5 family which has a vision encoder that triggers SYCL at import time in the registry subprocess.
Suggested fix:
Either:

Make the qwen3_vl import in qwen3_5.py lazy (inside the class that needs it, not at module level), or
Pass the full environment to the registry subprocess, or
Add a language_model_only check before importing the vision module in qwen3_5.py

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.