microsoft / microsoft/winml-cli
dinov2 / image-feature-extraction: all models pass wmk perf
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Summary
facebook/dinov2-giant fails with Error parsing message with type 'onnx.ModelProto' — the giant DINOv2 model exceeds the protobuf 2GB limit. Smaller variants (small, base, large) all pass.
Eval Results (2026-03-11)
| Status | Model | Task | Error |
|---|---|---|---|
| PASS | facebook/dinov2-small | image-feature-extraction | ✓ |
| PASS | facebook/dinov2-base | image-feature-extraction | ✓ |
| FAIL | facebook/dinov2-large | image-feature-extraction | TIMEOUT (600s) |
| FAIL | facebook/dinov2-giant | image-feature-extraction | Error parsing message with type 'onnx.ModelProto' |
| PASS | facebook/dino-vitb16 | image-feature-extraction | ✓ |
| PASS | facebook/dino-vits16 | image-feature-extraction | ✓ |
| PASS | StanfordAIMI/dinov2-base-xray-224 | image-feature-extraction | ✓ |
| PASS | microsoft/rad-dino | image-feature-extraction | ✓ |
2/8 fail — only large/giant variants.
Root Cause
dinov2-giant (~1.1B params) generates an ONNX file exceeding 2GB. Same root cause as xlm-roberta (#429) — requires save_as_external_data=True during export.
dinov2-large (~307M params) times out — may be a compilation timeout on the larger attention graph.
Current State
- No
dinov2.pyinmodelkit/models/hf/— relies on Optimum defaults - Small/base/xray/rad-dino all pass → export path is functional for smaller models
- External data format not enabled for large models
Desired State
All DINOv2 variants pass wmk perf, including dinov2-giant.
Acceptance Criteria
-
facebook/dinov2-giantpasseswmk perf -
facebook/dinov2-largepasseswmk perf - Existing small/base/dino-vitb16/dino-vits16 variants continue to pass
- Fix is size-based (universal) — not dinov2-specific (CLAUDE.md Cardinal Rule #1)
-
uv run pytest tests/passes (CLAUDE.md Cardinal Rule #3)
Technical Notes
- Fix: enable
use_external_data_format=Truefor exports above ONNX size threshold — coordinate with #429 (xlm-roberta) as the same fix applies - For
dinov2-largeTIMEOUT: may need an increased timeout for larger ViT models, or optimization to speed up QNN compilation for large attention graphs - This issue will be automatically resolved once the shared external-data-format fix from #429 is implemented
Related Files
modelkit/export/config.py—WinMLExportConfig, check foruse_external_data_formatfieldeval_results/2026-03-11/models/facebook__dinov2-giant__image-feature-extraction/result.json
References
- Blocked by / coordinate with: #429 (xlm-roberta ONNX size fix — same root cause and fix)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with modelkit/export/config.py and inspect WinMLExportConfig for external-data support, then review issue #429 and the referenced result.json. Run the DINOv2 wmk perf evaluations and uv run pytest tests/. Done means large and giant pass while the existing smaller variants remain passing and the fix is size-based.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100