microsoft / microsoft/winml-cli
SA: Coverage report investigation — OV NPU (2026-04-03)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Overview
Investigate regressions and unknown ops surfaced in the OpenVINO NPU SA coverage report (2026-04-03 run). Report: https://icy-moss-029643d00.6.azurestaticapps.net/sa_e2e_coverage/OpenVINOExecutionProvider_NPU/0403/sa_eval_report.html
Summary stats: Avg SUPPORTED (Pre): 95.5% → (Post): 95.5% | Avg delta: -0.1% | Avg UNKNOWN: 27 | All-SUPPORTED (Post): 99.1%
Regressions (Post < Pre)
| Model | Task | Pre | Post | Delta |
|---|---|---|---|---|
| microsoft/swin-large-patch4-window7-224 | image-classification | 68.8% | 66.7% | -2.1% |
Partial op patterns involved: Transpose
Unknown Ops
| Model | Unknown Count | Op Types |
|---|---|---|
| microsoft/deberta-xlarge-mnli | 1 | Slice |
| microsoft/swin-large-patch4-window7-224 | 4 | Add, Pad, Reshape, Slice |
| PekingU/rtdetr_r50vd_coco_o365 | 10 | GridSample, Tile, TopK, Add, Concat, Div, Mul, Reshape, Slice, Sub |
| PekingU/rtdetr_r101vd_coco_o365 | 10 | GridSample, Tile, TopK, Add, Concat, Div, Mul, Reshape, Slice, Sub |
| PekingU/rtdetr_v2_r18vd | 8 | GridSample, Tile, TopK, Add, Mul, Reshape, Slice, Sub |
Acceptance Criteria
- Root cause identified for swin-large regression (-2.1%) on OV NPU
-
Sliceclassified correctly for deberta-xlarge-mnli and swin on OV -
GridSample,Tile,TopKadded to OV coverage DB or flagged as unsupported - Next report shows 0 regressions and 0 unknown ops for the above models
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked OpenVINO NPU SA coverage report for the 2026-04-03 run and compare the pre/post results for swin-large-patch4-window7-224. Investigate the listed partial and unknown ops across the named models, then verify their classification or coverage status. Done means the regression root cause is identified, required ops are covered or explicitly unsupported, and a subsequent report has no listed regressions or unknown ops.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100