intel / intel/llm-scaler

[Bug] GLM-4.7-Flash MTP fails with sample_recovered_tokens_kernel build error on Intel XPU (ZE_RESULT_ERROR_MODULE_BUILD_FAILURE)

Open
#344 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

### Environment
- **Docker Image**: `intel/llm-scaler-vllm:0.14.0-b8.1`
- **Model**: GLM-4.7-Flash (official)
- **Hardware**: Intel XPU (Arc / Flex series, TP=2)
- **Key Flags**:
- `--tensor-parallel-size 2 --enable-expert-parallel`
- `--quantization fp8 --kv-cache-dtype fp8`
- `--speculative-config.method mtp --speculative-config.num_speculative_tokens 1`
- `--enforce-eager --max-num-seqs 8 --max-model-len 32768`

### Problem Description
When enabling **MTP (Multi-Token Prediction)**, the service starts successfully, but fails during inference with the following kernel compilation error on Worker_TP1:

L0 build module failed. Log:
warning: [RetryManager] Start recompilation of the kernel
in kernel: 'sample_recovered_tokens_kernel'
error: total scratch space exceeds HW supported limit for kernel sample_recovered_tokens_kernel: 292416 bytes (max permitted PTSS 262144 bytes)
error: backend compiler failed build.
Error during Intel loadBinary: ZE_RESULT_ERROR_MODULE_BUILD_FAILURE
RuntimeError: ZE_RESULT_ERROR_MODULE_BUILD_FAILURE

This triggers the shm_broadcast timeout:
> No available shared memory broadcast block found in 60 seconds.

### What I Tried
- Increased `VLLM_RPC_TIMEOUT=120`
- Set `TRITON_INTEL_GRF_MODE=256`
- Enabled `TRITON_INTEL_ENABLE_ADDRESS_PAYLOAD_OPTIMIZATION=1`
- `SYCL_CACHE_PERSISTENT=1`, `NEOReadDebugKeys=1`, etc.
- Various `--gpu-memory-utilization` and `--max-num-seqs` values

**Result**: The kernel still fails to compile when MTP is enabled.
**When MTP is disabled**, inference works normally.

Contributor guide

Open the contributing guide

Research direction

Reproduce with the listed Docker image, GLM-4.7-Flash, Intel XPU TP=2, and MTP enabled, then compare with MTP disabled. Start from sample_recovered_tokens_kernel and the PTSS/scratch-space compiler error; done means the kernel builds successfully and inference no longer reaches the shm_broadcast timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker
Domain
ai-infra-agents, build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.