microsoft / microsoft/winml-cli

Add --report flag to winml build for agent-friendly JSON output

Open
#443 0 comments 0 reactions 1 assignee View on GitHub

@hi-brenda is already working on this.

Since May 7, 2026.

dev experience P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Summary

Add a --report <path> flag to winml build that writes a structured JSON file capturing the full build result — stage-by-stage status, timings, artifact paths, and analyze findings — so downstream agents and automation pipelines can consume build outcomes without parsing Rich console output.

Motivation

winml build currently has no machine-readable output. All result data (stage timings, artifact paths, analyze node counts, autoconf patterns) is rendered through the Rich console and discarded. This blocks:

  • Agent pipelines (e.g., winml flow) from reading build results to decide next steps
  • CI/CD from asserting on specific stages without scraping terminal output
  • Tooling integrations (foundryTK, VS Code extensions) from surfacing build summaries to users

The only partial exception is module mode, which writes a minimal module_summary.json — but it omits per-stage details and is not available in single-model mode.

Proposed Solution

Flag
winml build -c config.json -m ProsusAI/finbert -o output/ --report output/build_report.json

If --report is omitted, behavior is unchanged (no file written).

Report schema
{
  "schema_version": "1",
  "timestamp": "2026-05-06T10:23:00Z",
  "model_id": "ProsusAI/finbert",
  "config_file": "config.json",
  "output_dir": "output/",
  "success": true,
  "total_elapsed_s": 94.2,
  "stages": [
    {
      "name": "export",
      "status": "success",
      "elapsed_s": 18.4,
      "artifact": { "path": "output/export.onnx", "size_bytes": 438291456 }
    },
    {
      "name": "optimize",
      "status": "success",
      "elapsed_s": 31.7,
      "artifact": { "path": "output/optimized.onnx", "size_bytes": 412680192 },
      "analyze": {
        "iterations": 2,
        "converged": true,
        "autoconf_patterns": ["disable_layer_norm_fusion"],
        "node_counts": { "supported": 312, "partial": 4, "unsupported": 0 }
      }
    },
    {
      "name": "quantize",
      "status": "skipped",
      "reason": "QDQ nodes already present"
    },
    {
      "name": "compile",
      "status": "success",
      "elapsed_s": 44.1,
      "artifact": { "path": "output/compiled.onnx", "size_bytes": 109051904 }
    }
  ],
  "final_artifact": { "path": "output/model.onnx", "size_bytes": 109051904 },
  "reused": false
}

Failure case adds a top-level "error" field with the message, and the failed stage has "status": "failed".

Implementation notes
  • stage_timings in _run_single_build already collects (name, elapsed | None) — extend it to carry status, artifact path/size, and analyze details
  • Write the report in the finally block of _run_single_build so partial results are captured even on failure
  • Module mode should extend the existing module_summary.json with the same per-stage detail rather than a separate file

Acceptance Criteria

  • --report <path> flag added to winml build
  • JSON written on both success and failure (partial results on failure)
  • Report includes: schema_version, timestamp, model_id, config_file, output_dir, success, total_elapsed_s, stages, final_artifact, reused
  • Each stage entry includes: name, status (success/skipped/failed), elapsed_s, artifact (path + size), and for optimize: analyze sub-object
  • Module mode extends module_summary.json with per-stage detail
  • Unit tests cover report schema for success, skip, failure, and reuse cases
  • --report omitted → no behavior change (no file written, no performance impact)

Alternatives Considered

  • Parse console output: Fragile, breaks on any Rich formatting change, not suitable for agents.
  • Always write a report to output dir: Adds implicit side-effects; opt-in flag is cleaner.

Additional Context

The --report output is the intended data source for winml flow (see #442) so each pipeline step can surface a structured summary before prompting the user to continue or skip. It is also a natural data feed for the foundryTK Template UI to display build progress and results without re-running the build.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.