[Bug] GLM-4.7-Flash MTP fails with sample_recovered_tokens_kernel build error on Intel XPU (ZE_RESULT_ERROR_MODULE_BUILD_FAILURE)
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
### Environment
- **Docker Image**: `intel/llm-scaler-vllm:0.14.0-b8.1`
- **Model**: GLM-4.7-Flash (official)
- **Hardware**: Intel XPU (Arc / Flex series, TP=2)
- **Key Flags**:
- `--tensor-parallel-size 2 --enable-expert-parallel`
- `--quantization fp8 --kv-cache-dtype fp8`
- `--speculative-config.method mtp --speculative-config.num_speculative_tokens 1`
- `--enforce-eager --max-num-seqs 8 --max-model-len 32768`
### Problem Description
When enabling **MTP (Multi-Token Prediction)**, the service starts successfully, but fails during inference with the following kernel compilation error on Worker_TP1:
L0 build module failed. Log:
warning: [RetryManager] Start recompilation of the kernel
in kernel: 'sample_recovered_tokens_kernel'
error: total scratch space exceeds HW supported limit for kernel sample_recovered_tokens_kernel: 292416 bytes (max permitted PTSS 262144 bytes)
error: backend compiler failed build.
Error during Intel loadBinary: ZE_RESULT_ERROR_MODULE_BUILD_FAILURE
RuntimeError: ZE_RESULT_ERROR_MODULE_BUILD_FAILURE
This triggers the shm_broadcast timeout:
> No available shared memory broadcast block found in 60 seconds.
### What I Tried
- Increased `VLLM_RPC_TIMEOUT=120`
- Set `TRITON_INTEL_GRF_MODE=256`
- Enabled `TRITON_INTEL_ENABLE_ADDRESS_PAYLOAD_OPTIMIZATION=1`
- `SYCL_CACHE_PERSISTENT=1`, `NEOReadDebugKeys=1`, etc.
- Various `--gpu-memory-utilization` and `--max-num-seqs` values
**Result**: The kernel still fails to compile when MTP is enabled.
**When MTP is disabled**, inference works normally.
Contributor guide
Research direction
Reproduce with the listed Docker image, GLM-4.7-Flash, Intel XPU TP=2, and MTP enabled, then compare with MTP disabled. Start from sample_recovered_tokens_kernel and the PTSS/scratch-space compiler error; done means the kernel builds successfully and inference no longer reaches the shm_broadcast timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker
- Domain
- ai-infra-agents, build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100