intel / intel/llm-scaler

MiniCPM-V 4.5 regression on Intel LLM Scaler `0.14.0-b8.2`

Open
#380 0 comments 0 reactions 1 assignee Claimed by @montaguelhz View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

I appear to be hitting a regression with MiniCPM-V 4.5 on Intel XPU using `intel/llm-scaler-vllm:0.14.0-b8.2`.

The same MiniCPM-V 4.5 launcher/topology works correctly after rolling back to the prior Scaler build, but fails on `0.14.0-b8.2`.

### Hardware / topology

System has multiple Intel Battlemage GPUs. For this MiniCPM-V test I am using two Arc Pro B60 GPUs:

```text
renderD133 -> Intel Battlemage G21 / Arc Pro B60
renderD134 -> Intel Battlemage G21 / Arc Pro B60
ZE_AFFINITY_MASK=5,6
ONEAPI_DEVICE_FILTER=level_zero:gpu:5,level_zero:gpu:6
tensor-parallel-size=2
````

The two devices are visible correctly inside the container:

```text
xpu available: True
device count: 2
0 Intel(R) Arc(TM) Pro B60 Graphics
1 Intel(R) Arc(TM) Pro B60 Graphics
```

### Working version

MiniCPM-V 4.5 works after rolling back from:

```text
intel/llm-scaler-vllm:0.14.0-b8.2
```

to the prior Scaler image.

### Failing version

```text
intel/llm-scaler-vllm:0.14.0-b8.2
```

### Launch shape

The relevant launch settings are:

```bash
--model /llm/models/MiniCPM-V-4_5
--served-model-name MiniCPM-V-4_5
--trust-remote-code
--dtype bfloat16
--host 0.0.0.0
--port 8002
--gpu-memory-utilization 0.70
--enable-prefix-caching
--max-model-len 49152
--max-num-batched-tokens 16384
--max-num-seqs 1
--swap-space 16
--tensor-parallel-size 2
--block-size 64
--enforce-eager
--disable-log-requests
```

### Failure

On `0.14.0-b8.2`, startup fails during vLLM multimodal profiling / MiniCPMVProcessor initialization.

The key error is:

```text
AttributeError: CachedTokenizersBackend has no attribute im_start_id
```

Full relevant stack includes:

```text
File ".../processing_minicpmv.py", line 115, in _convert
start_cond = (input_ids == self.tokenizer.im_start_id) | (input_ids == self.tokenizer.slice_start_id)

AttributeError: CachedTokenizersBackend has no attribute im_start_id
```

Then vLLM wraps it as:

```text
ValueError: Failed to apply MiniCPMVProcessor on data={
'text': ['(./)'],
'images': [[]]
} with kwargs={'truncation': False}
```

The failure occurs before the API server comes up.

### What I ruled out

This does **not** appear to be:

* GPU visibility
* bad render node mapping
* ASPEED onboard video confusion
* VRAM pressure
* CPU/NUMA affinity
* model path issue

The same GPU pair and launcher topology work after reverting the Scaler version.

### Suspected cause

It looks like `0.14.0-b8.2` changed the tokenizer / cached tokenizer path used during MiniCPM-V multimodal profiling. MiniCPM’s processor expects tokenizer attributes such as:

```text
im_start_id
slice_start_id
```

but the tokenizer object being passed is:

```text
CachedTokenizersBackend
```

which does not expose those attributes.

### Workaround

Rolling back from `intel/llm-scaler-vllm:0.14.0-b8.2` to the prior Scaler build restores MiniCPM-V 4.5 startup.

### Request

Could you check whether the tokenizer backend / MiniCPMVProcessor path changed in `0.14.0-b8.2`, specifically for MiniCPM-V 4.5 multimodal profiling?

It looks like the processor is receiving a cached tokenizer wrapper instead of the tokenizer object expected by MiniCPM’s remote processor code.

```

The important bit is the regression is clean: same topology, same model, same hardware, prior Scaler works, `b8.2` breaks on `CachedTokenizersBackend`. :contentReference[oaicite:0]{index=0}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.