intel / intel/AI-Playground

NPU-supported INT4/NF4 OpenVINO models fail NPU compilation

Open
#564 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
979
Forks
132
Avg merge
9h 23m
Merged PRs (30d)
8

Description

## Summary

AI Playground 3.1.2-beta marks some OpenVINO LLMs as `npuSupport: true`, but the bundled OpenVINO Model Server fails to compile them for the selected Intel NPU. This is separate from the Windows OVMS embedded-Python loader issue reported in #563.

## Affected models

Both locally downloaded models that `resources/models.json` marks as NPU-supported fail identically:
- `OpenVINO/DeepSeek-R1-Distill-Qwen-7B-nf4-ov`
- `OpenVINO/Qwen3-8B-int4-cw-ov`

## Actual behavior

OVMS starts, initializes its embedded Python, detects CPU/GPU/NPU, begins model loading, then exits with code 1:

```text
[vpux-compiler] Network has 4-bit output, which is not yet supported
[vpux-compiler] Failed Pass ForbidFourBitOutputs
Compilation failed. vclAllocatedExecutableCreate3 result: 0x78000004
```

OVMS then reports `LOADING_PRECONDITION_FAILED` and `The LLM Node resource initialization failed`.

## Control test

The same downloaded `OpenVINO/Qwen3-8B-int4-cw-ov` model reaches `AVAILABLE` on CPU when started without the NPU-only `MAX_PROMPT_LEN` option. This confirms that the model download and OVMS package are sound; the failure is specific to NPU compilation.

## Environment

- AI Playground `3.1.2-beta`
- OVMS `2026.3.0`, release tag `8022ddae3`
- Intel NPU driver `32.0.100.4778` (April 28, 2026), device status `OK`
- Intel Graphics driver `32.0.101.8801` (May 12, 2026)

## Expected behavior

Models advertised as NPU-supported should compile and load on a supported Intel NPU. If these model formats require a newer NPU compiler/runtime or driver, they should not be marked as NPU-supported until that dependency is enforced or documented.

## Requested action

Please review the NPU compatibility metadata for the affected INT4/NF4 models and the bundled NPU compiler/runtime compatibility. Either:
1. ship a compatible NPU stack; or
2. remove/disable `npuSupport` for models that the current stack cannot compile; or
3. enforce and clearly report the required Intel NPU driver/runtime version.

Contributor guide

Open the contributing guide

Research direction

Start with resources/models.json to inspect the npuSupport metadata for the two affected OpenVINO models, then reproduce their OVMS startup on NPU and compare it with the documented CPU control test. Review the bundled OVMS and NPU compiler/runtime compatibility; done means the metadata and supported-driver behavior accurately match whether the models reach AVAILABLE.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.