canonical / canonical/inference-snaps
`intel-npu` engine fails with a large image
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 50
- Forks
- 6
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 7
Description
With a larger image, the intel-npu engine fails, responding with:
```
{'error': 'Mediapipe execution failed. MP status - INVALID_ARGUMENT: CalculatorGraph::Run() failed: \nCalculator::Process() for node "LLMExecutor" failed: Request processing failed, check its correctness.'}
```
The server logs report the real reason:
```
VLM pipeline on NPU may only process input embeddings up to 1024 tokens. 1165 is passed.
Set the "MAX_PROMPT_LEN" config option to increase the limit.
```
Intel has mentioned this before:
> for VLMs on NPU it is easy to get a lot of input tokens so you may need to set MAX_PROMPT_LEN as described here for LLMs https://docs.openvino.ai/2025/openvino-workflow-generative/inference-with-genai/inference-with-genai-on-npu.html .
Increasing the maximum prompt length does not work, as ovms does not parse the option correctly. See [upstream bug report](https://github.com/openvinotoolkit/model_server/issues/3703).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported intel-npu failure and the linked upstream model_server issue, then inspect how MAX_PROMPT_LEN is passed to the VLM pipeline. Confirm whether the upstream parsing problem is resolved and whether a large-image request can process more than 1024 tokens; the issue is done when this path works or its upstream dependency is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100