microsoft / microsoft/winml-cli

P1-MODEL-003: 20 ISV Models — Architecture Analysis Report

Open
#149 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model / task scale P1 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Summary

Engage ISV partners to collect 20 partner models, analyze their architecture and export compatibility, and produce a structured analysis report covering architecture summary, export status, EP compatibility, and deployment gaps.

Context

ISV partner models are production models from software vendors — often fine-tuned, quantized, or using custom operators that standard pipelines don't handle. This analysis report is the primary deliverable (not full pipeline support).

This combines:

  • plans/release/0315_release_plan/P1_CHECKLIST.md: P1-MODEL-001 (ISV Outreach), P1-MODEL-002 (ISV Analysis), P1-MODEL-003 (ISV Custom Support), P1-MODEL-004 (ISV Validation)
  • plans/release/0501_release_plan/P0_CHECKLIST.md: P1-MODEL-003

Current State

  • No ISV models collected or analyzed
  • No ISV partner engagement initiated

Desired State

  • 20 ISV models collected from 3-5 ISV partners
  • Per-model analysis report: architecture, custom ops, export readiness, EP compatibility
  • Deployment gap analysis: what's blocking production deployment on NPU/GPU targets
  • Best-effort: attempt E2E pipeline; document failures with root cause

Acceptance Criteria

  • 3-5 ISV partners engaged; 20 model samples collected
  • Architecture analysis complete for all 20 models (model structure, op types, custom ops)
  • Export attempt (ONNX) with failure root cause documented for each model
  • EP compatibility analysis run on successfully exported models
  • Analysis report published per model: architecture summary, export status, EP compat, gaps
  • Deployment gap summary: top 5 blockers across all ISV models

Technical Notes

  • Scope: analysis report is the primary deliverable — full pipeline support is best-effort only
  • ISV categories: fine-tuned models, quantized production models, custom op models, hybrid architectures
  • If no partner access, use public representative models as stand-ins
  • Custom op handling: document what custom ops are present; implementation is out of scope

Related Files

  • plans/release/0501_release_plan/model-scale.md — ISV section
  • plans/release/0315_release_plan/P1_CHECKLIST.md — P1-MODEL-001~004

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading plans/release/0501_release_plan/model-scale.md and the P1-MODEL-001~004 sections in plans/release/0315_release_plan/P1_CHECKLIST.md. Establish access to 3-5 partners or representative public models, then track the 20 analyses against the acceptance criteria. Done means each model has an architecture, export, compatibility, and gap report, plus a top-five deployment blocker summary.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.