intel / intel/llm-scaler

AWQ quantized models fail with zeMemOpenIpcHandle ZE_RESULT_ERROR_INVALID_ARGUMENT on dual Arc Pro B70 (Linux + llm-scaler vLLM, tp=2)

Open
#398 2 comments 0 reactions 1 assignee Claimed by @liu-shaojun View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Environment
- GPU: 2x Intel Arc Pro B70 (64GB VRAM total, tp=2)
- OS: Ubuntu 26.04
- Kernel: 7.0.0-15-generic
- Intel GPU Driver: 26.05.037020 (via clinfo)
- Stack: llm-scaler-vllm (Docker)

## Error
zeMemOpenIpcHandle, code: ZE_RESULT_ERROR_INVALID_ARGUMENT
RuntimeError: Engine core initialization failed. Failed core proc(s): {'EngineCore_DP0': 1}

## Model tested
deepseek-ai/DeepSeek-R1-70B-AWQ

## Steps to reproduce
vllm serve /llm/models/DeepSeek-R1-70B-AWQ --dtype=float16
--enforce-eager --port 8008 -tp=2 --gpu-memory-util=0.90
--max-model-len=8192 --allow-deprecated-quantization

## What was tried
- CCL_ATL_TRANSPORT=ofi → same error
- CCL_TOPO_FABRIC_VERTEX_CONNECTION_CHECK=0 → same error
- --disable-custom-all-reduce → same error

## Expected behavior
AWQ model loads and runs inference correctly with tp=2 on dual B70.

## Actual behavior
Fails consistently at zeMemOpenIpcHandle during IPC memory sharing
between the two GPUs. Non-AWQ models (FP16) work correctly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.