NPU-supported INT4/NF4 OpenVINO models fail NPU compilation
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 132
- Avg merge
- 9h 23m
- Merged PRs (30d)
- 8
Description
## Summary
AI Playground 3.1.2-beta marks some OpenVINO LLMs as `npuSupport: true`, but the bundled OpenVINO Model Server fails to compile them for the selected Intel NPU. This is separate from the Windows OVMS embedded-Python loader issue reported in #563.
## Affected models
Both locally downloaded models that `resources/models.json` marks as NPU-supported fail identically:
- `OpenVINO/DeepSeek-R1-Distill-Qwen-7B-nf4-ov`
- `OpenVINO/Qwen3-8B-int4-cw-ov`
## Actual behavior
OVMS starts, initializes its embedded Python, detects CPU/GPU/NPU, begins model loading, then exits with code 1:
```text
[vpux-compiler] Network has 4-bit output, which is not yet supported
[vpux-compiler] Failed Pass ForbidFourBitOutputs
Compilation failed. vclAllocatedExecutableCreate3 result: 0x78000004
```
OVMS then reports `LOADING_PRECONDITION_FAILED` and `The LLM Node resource initialization failed`.
## Control test
The same downloaded `OpenVINO/Qwen3-8B-int4-cw-ov` model reaches `AVAILABLE` on CPU when started without the NPU-only `MAX_PROMPT_LEN` option. This confirms that the model download and OVMS package are sound; the failure is specific to NPU compilation.
## Environment
- AI Playground `3.1.2-beta`
- OVMS `2026.3.0`, release tag `8022ddae3`
- Intel NPU driver `32.0.100.4778` (April 28, 2026), device status `OK`
- Intel Graphics driver `32.0.101.8801` (May 12, 2026)
## Expected behavior
Models advertised as NPU-supported should compile and load on a supported Intel NPU. If these model formats require a newer NPU compiler/runtime or driver, they should not be marked as NPU-supported until that dependency is enforced or documented.
## Requested action
Please review the NPU compatibility metadata for the affected INT4/NF4 models and the bundled NPU compiler/runtime compatibility. Either:
1. ship a compatible NPU stack; or
2. remove/disable `npuSupport` for models that the current stack cannot compile; or
3. enforce and clearly report the required Intel NPU driver/runtime version.
Contributor guide
Research direction
Start with resources/models.json to inspect the npuSupport metadata for the two affected OpenVINO models, then reproduce their OVMS startup on NPU and compare it with the documented CPU control test. Review the bundled OVMS and NPU compiler/runtime compatibility; done means the metadata and supported-driver behavior accurately match whether the models reach AVAILABLE.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100