intel / intel/llm-scaler

Intel llm-scaler vLLM crashes on Gemma 4 due to k_descale stride assertion

Open
#532 4 comments 0 reactions 1 assignee Claimed by @Wesley-Du View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

### Environment

* GPU: Intel Arc Pro B50 16 GB
* Image: `intel/llm-scaler-vllm:0.21.0-b1`
* `vllm-xpu-kernels`: `0.1.8.3.dev0+g3cab97a.d20260709`
* Model: `google/gemma-4-12B-it-qat-w4a16-ct`

### Problem

The model loads successfully, but the first inference request crashes with:

```text
AssertionError: k_descale must be view of single float32 scalar tensor
```

The supplied tensor is:

```text
shape=(1, 1)
dtype=torch.float32
stride=(1, 1)
device=xpu:0
```

It contains exactly one valid FP32 value, but the XPU FlashAttention wrapper requires a zero-stride scalar-backed view.

### Confirmed workaround

Normalizing the tensor before the assertion fixes inference:

```python
original_shape = k_descale.shape
scalar = k_descale.reshape(-1)[0].to(dtype=torch.float32)
k_descale = scalar.reshape(()).expand(original_shape)
```

The same normalization was applied to `v_descale`.

After this small wrapper patch, Gemma 4 12B QAT generates successfully on the B50.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.