MiniCPM-V 4.5 regression on Intel LLM Scaler `0.14.0-b8.2`
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
I appear to be hitting a regression with MiniCPM-V 4.5 on Intel XPU using `intel/llm-scaler-vllm:0.14.0-b8.2`.
The same MiniCPM-V 4.5 launcher/topology works correctly after rolling back to the prior Scaler build, but fails on `0.14.0-b8.2`.
### Hardware / topology
System has multiple Intel Battlemage GPUs. For this MiniCPM-V test I am using two Arc Pro B60 GPUs:
```text
renderD133 -> Intel Battlemage G21 / Arc Pro B60
renderD134 -> Intel Battlemage G21 / Arc Pro B60
ZE_AFFINITY_MASK=5,6
ONEAPI_DEVICE_FILTER=level_zero:gpu:5,level_zero:gpu:6
tensor-parallel-size=2
````
The two devices are visible correctly inside the container:
```text
xpu available: True
device count: 2
0 Intel(R) Arc(TM) Pro B60 Graphics
1 Intel(R) Arc(TM) Pro B60 Graphics
```
### Working version
MiniCPM-V 4.5 works after rolling back from:
```text
intel/llm-scaler-vllm:0.14.0-b8.2
```
to the prior Scaler image.
### Failing version
```text
intel/llm-scaler-vllm:0.14.0-b8.2
```
### Launch shape
The relevant launch settings are:
```bash
--model /llm/models/MiniCPM-V-4_5
--served-model-name MiniCPM-V-4_5
--trust-remote-code
--dtype bfloat16
--host 0.0.0.0
--port 8002
--gpu-memory-utilization 0.70
--enable-prefix-caching
--max-model-len 49152
--max-num-batched-tokens 16384
--max-num-seqs 1
--swap-space 16
--tensor-parallel-size 2
--block-size 64
--enforce-eager
--disable-log-requests
```
### Failure
On `0.14.0-b8.2`, startup fails during vLLM multimodal profiling / MiniCPMVProcessor initialization.
The key error is:
```text
AttributeError: CachedTokenizersBackend has no attribute im_start_id
```
Full relevant stack includes:
```text
File ".../processing_minicpmv.py", line 115, in _convert
start_cond = (input_ids == self.tokenizer.im_start_id) | (input_ids == self.tokenizer.slice_start_id)
AttributeError: CachedTokenizersBackend has no attribute im_start_id
```
Then vLLM wraps it as:
```text
ValueError: Failed to apply MiniCPMVProcessor on data={
'text': ['(./)'],
'images': [[]]
} with kwargs={'truncation': False}
```
The failure occurs before the API server comes up.
### What I ruled out
This does **not** appear to be:
* GPU visibility
* bad render node mapping
* ASPEED onboard video confusion
* VRAM pressure
* CPU/NUMA affinity
* model path issue
The same GPU pair and launcher topology work after reverting the Scaler version.
### Suspected cause
It looks like `0.14.0-b8.2` changed the tokenizer / cached tokenizer path used during MiniCPM-V multimodal profiling. MiniCPM’s processor expects tokenizer attributes such as:
```text
im_start_id
slice_start_id
```
but the tokenizer object being passed is:
```text
CachedTokenizersBackend
```
which does not expose those attributes.
### Workaround
Rolling back from `intel/llm-scaler-vllm:0.14.0-b8.2` to the prior Scaler build restores MiniCPM-V 4.5 startup.
### Request
Could you check whether the tokenizer backend / MiniCPMVProcessor path changed in `0.14.0-b8.2`, specifically for MiniCPM-V 4.5 multimodal profiling?
It looks like the processor is receiving a cached tokenizer wrapper instead of the tokenizer object expected by MiniCPM’s remote processor code.
```
The important bit is the regression is clean: same topology, same model, same hardware, prior Scaler works, `b8.2` breaks on `CachedTokenizersBackend`. :contentReference[oaicite:0]{index=0}
```
Contributor guide
Assessment
This issue has not been assessed yet.