canonical / canonical/inference-snaps

`intel-npu` engine fails with a large image

Open
#91 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

snap/qwen-vl
Dominant language
No language data
Stars
50
Forks
6
Avg merge
1d 20h
Merged PRs (30d)
7

Description

With a larger image, the intel-npu engine fails, responding with:
```
{'error': 'Mediapipe execution failed. MP status - INVALID_ARGUMENT: CalculatorGraph::Run() failed: \nCalculator::Process() for node "LLMExecutor" failed: Request processing failed, check its correctness.'}
```

The server logs report the real reason:
```
VLM pipeline on NPU may only process input embeddings up to 1024 tokens. 1165 is passed.
Set the "MAX_PROMPT_LEN" config option to increase the limit.
```

Intel has mentioned this before:
> for VLMs on NPU it is easy to get a lot of input tokens so you may need to set MAX_PROMPT_LEN as described here for LLMs https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-genai/inference-with-genai-on-npu.html .

Increasing the maximum prompt length does not work, as ovms does not parse the option correctly. See [upstream bug report](https://github.com/openvinotoolkit/model_server/issues/3703).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported intel-npu failure and the linked upstream model_server issue, then inspect how MAX_PROMPT_LEN is passed to the VLM pipeline. Confirm whether the upstream parsing problem is resolved and whether a large-image request can process more than 1024 tokens; the issue is done when this path works or its upstream dependency is clearly documented.

Written by the indexing model from the issue text.

Assessment

Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.