intel / intel/llm-scaler

MinerU2.5-Pro-2605-1.2B outputs deterministic garbage on Arc Pro B70 (0.21.0-b1); MinerU2.5-2509-1.2B works

Open
#580 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Summary

`opendatalab/MinerU2.5-Pro-2605-1.2B` produces deterministic garbage output on Arc Pro B70 with `intel/llm-scaler-vllm:0.21.0-b1`, while the older `opendatalab/MinerU2.5-2509-1.2B` (the one listed in the supported-models table) works correctly on the exact same setup.

MinerU2.5-Pro-2605 is the model current MinerU releases deploy in production (it adds chart-to-table extraction and much better CJK glyph fidelity), so supporting it would let MinerU users move document-parsing workloads to Arc GPUs.

## Environment

- GPU: 2x Intel Arc Pro B70 (32GB, Battlemage), driver 26.27.x
- Host: Ubuntu 26.04, kernel 6.8
- Image: `intel/llm-scaler-vllm:0.21.0-b1` (digest as of 2026-07-30)
- Model: `opendatalab/MinerU2.5-Pro-2605-1.2B` (resolved architecture: `Qwen2VLForConditionalGeneration`)

## Reproduction

```bash
docker run -d --name mineru-vllm --net=host --device=/dev/dri --group-add video \
--shm-size=16g -v /path/to/models:/llm/models -e ZE_AFFINITY_MASK=0 \
intel/llm-scaler-vllm:0.21.0-b1 \
/llm/models/MinerU2.5-Pro-2605-1.2B \
--served-model-name MinerU2.5-Pro-2605-1.2B \
--dtype bfloat16 --enforce-eager --trust-remote-code \
--host 127.0.0.1 --port 18100 --max-model-len 8192 --gpu-memory-utilization 0.5

curl -s http://127.0.0.1:18100/v1/chat/completions -H "Content-Type: application/json" \
-d '{"model":"MinerU2.5-Pro-2605-1.2B","messages":[{"role":"user","content":"Hello, what can you do?"}],"max_tokens":50}'
```

## Observed

Deterministic multilingual token soup, e.g.:

```
� döneminde.sendMessage胱 btcمطار תיקредел-In:http Proposition Gerry基数沒有 pregnantcly三级协会会员旺盛_TRAN sweetness_limits[]);
```

Notes:

- The output is **byte-identical across runs and across `--dtype float16` / `--dtype bfloat16`**, so it does not look like nondeterministic numeric noise.
- With a real MinerU client workload (`mineru -b vlm-http-client`, image input), the layout-detection step returns `!!!!!!!!...` repeated until the length limit, so the failure affects both text-only and multimodal inputs.
- Generation speed is normal (~50 tok in ~4s incl. warmup), the server starts cleanly, and XPU kernels are engaged (`Final IR op priority ... xpu_kernels`).

## Expected

Coherent output, as produced by:

- the same model on CUDA (lmdeploy/turbomind on RTX 2080 Ti) — our production baseline, correct output;
- `opendatalab/MinerU2.5-2509-1.2B` on the **same** llm-scaler-vllm setup (only the model path changed) — correct markdown output end-to-end via the MinerU client, ~4.5s/page on a single B70.

Since 2509 works and Pro-2605 fails identically under both dtypes, this looks like an op/kernel or weight-loading incompatibility specific to the Pro-2605 checkpoint rather than a general Qwen2-VL problem.

Happy to run patches or collect additional logs on our B70 hosts if that helps.

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the supplied docker run command and the OpenAI-compatible /v1/chat/completions request, then compare it with MinerU2.5-2509-1.2B on the same B70 setup. Check both float16 and bfloat16 paths and the MinerU `vlm-http-client` workload; done means coherent text and multimodal output matching the working checkpoint and CUDA baseline.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, linux
Domain
computer-vision, devops, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.