Cannot run TP=2 across two Arc Pro B70 in intel/vllm:0.21.0-xpu - all CCL_ZE_IPC_EXCHANGE modes fail
- Dominant language
- Dockerfile
- Stars
- 1
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Tensor-parallel serving (--tensor-parallel-size 2) across two Intel Arc Pro B70 (Battlemage G31) GPUs fails inside the OMIX vLLM container intel/vllm:0.21.0-xpu (OMIX 0.3.0, vLLM 0.21.x, oneCCL 2021.17).
The model loads successfully on both GPUs (~19.4 GiB each) and the xccl process group initializes, but the very first cross-GPU all-reduce (during determine_available_memory → profile_run, at the vocab-parallel embedding) aborts with:
RuntimeError: oneCCL: ze_call.cpp:28 do_call: EXCEPTION:
ze error at zeMemOpenIpcHandle, code: ZE_RESULT_ERROR_INVALID_ARGUMENT
This is not a fd-transport problem: it reproduces identically with CCL_ZE_IPC_EXCHANGE=sockets and CCL_ZE_IPC_EXCHANGE=pidfd (two different fd-passing mechanisms that both fail at the same zeMemOpenIpcHandle call), and CCL_ZE_IPC_EXCHANGE=drmfd fails even earlier during device init. That points at the Level-Zero runtime being unable to open a peer GPU's IPC memory handle at all inside this container.
Reproduction
Both B70s are Level-Zero devices 0 and 1. The container is launched without overriding the entrypoint (so /opt/intel/oneapi/setvars.sh is sourced), passing vllm serve … as the command:
RENDER_GID=$(getent group render | cut -d: -f3); VIDEO_GID=$(getent group video | cut -d: -f3)
docker run --rm -it --name vllm-72b-omix \
--privileged --device /dev/dri \
--group-add "$RENDER_GID" --group-add "$VIDEO_GID" \
--ipc=host --shm-size=16g \
-e ZE_AFFINITY_MASK=0,1 -e ZES_ENABLE_SYSMAN=1 \
-e VLLM_WORKER_MULTIPROC_METHOD=spawn -e SYCL_UR_USE_LEVEL_ZERO_V2=0 \
-e CCL_ZE_IPC_EXCHANGE=sockets -e CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0 \
-e CCL_ATL_TRANSPORT=ofi \
-v ~/.cache/huggingface:/root/.cache/huggingface -p 8003:8000 \
intel/vllm:0.21.0-xpu \
vllm serve Qwen/Qwen2.5-72B-Instruct-GPTQ-Int4 \
--tensor-parallel-size 2 --dtype float16 --enforce-eager \
--block-size 64 --max-model-len 8192 --gpu-memory-utilization 0.9 \
--host 0.0.0.0 --port 8000
Expected behavior
The engine completes memory profiling and reaches Application startup complete., serving the model over the OpenAI-compatible API — as it does on intel/vllm:0.17.0-xpu.
Actual behavior
Weights load on both workers, then the first all-reduce fails:
(Worker_TP0) INFO ... Model loading took 19.4 GiB memory and 12.36 seconds
(Worker_TP1) ERROR ... WorkerProc hit an exception.
File ".../vllm/v1/worker/gpu_worker.py", line 392, in determine_available_memory
self.model_runner.profile_run()
...
File ".../vllm/model_executor/layers/vocab_parallel_embedding.py", line 489, in forward
output = tensor_model_parallel_all_reduce(output_parallel)
...
File ".../vllm/distributed/device_communicators/xpu_communicator.py", line 46, in all_reduce
dist.all_reduce(output, group=self.device_group)
...
RuntimeError: oneCCL: ze_call.cpp:28 do_call: EXCEPTION:
ze error at zeMemOpenIpcHandle, code: ZE_RESULT_ERROR_INVALID_ARGUMENT
Matrix of CCL_ZE_IPC_EXCHANGE modes tried
All other flags identical; only CCL_ZE_IPC_EXCHANGE changed:
Contributor guide
Research direction
Reproduce the two-GPU command with the three CCL_ZE_IPC_EXCHANGE modes and compare where each failure occurs. Start at vllm/v1/worker/gpu_worker.py line 392, model_executor/layers/vocab_parallel_embedding.py line 489, and distributed/device_communicators/xpu_communicator.py line 46. Done means memory profiling completes and the server reaches Application startup complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- ai-infra-agents, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100