microsoft / microsoft/winml-cli

feat: QDQ ↔ QLinear bidirectional conversion with EP compatibility

Open
#65 1 comment 0 reactions 1 assignee View on GitHub

@tezheng is already working on this.

Since Mar 31, 2026.

dev experience feature scale GPU NPU P1 QDQ triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Summary

Add bidirectional conversion between QDQ and QLinear quantization formats, with automatic format selection during compilation based on the target Execution Provider.

Background

Two ONNX quantization formats exist:

  • QDQ (QuantizeLinear + DequantizeLinear nodes): Preserves float graph structure, wraps ops with Q/DQ pairs
  • QLinear (QLinearConv, QLinearMatMul, etc.): Fused quantized operators that consume/produce integer tensors directly

Different EPs require different formats.

EP Compatibility Matrix

Execution Provider QDQ QLinear Notes
CPUExecutionProvider Not supported Supported ORT CPU kernels expect QLinear ops
QNNExecutionProvider (NPU) Supported Not supported QNN SDK consumes QDQ format
CUDAExecutionProvider (GPU) TBD TBD Needs investigation
DirectMLExecutionProvider TBD TBD Needs investigation

Design

1. Bidirectional converter
  • QDQ → QLinear: Fuse Q/DQ pairs around standard ops into their QLinear equivalents (e.g., Conv + surrounding Q/DQ → QLinearConv)
  • QLinear → QDQ: Decompose QLinear ops back into float ops wrapped with Q/DQ pairs
2. EP compatibility registry
  • Map each EP to its supported quantization format(s)
  • Queryable at compile time: `get_supported_quant_format(ep_name) -> "qdq" | "qlinear" | "both"`
3. Auto-conversion in compilation stage

During the compile step, detect the model's current quantization format and the target EP:

  • Model is QDQ + target is CPU → auto-convert QDQ → QLinear before compilation
  • Model is QLinear + target is NPU → auto-convert QLinear → QDQ before compilation
  • Model format matches EP → no conversion needed
4. Integration point
  • New conversion step in the compile stage (before EP context creation)
  • Logged as a sub-step in the build manifest
  • Conversion should be idempotent (running it on an already-correct format is a no-op)

Acceptance Criteria

  • QDQ → QLinear graph transformation implemented
  • QLinear → QDQ graph transformation implemented
  • EP compatibility registry with format lookup
  • Compile stage auto-detects model format and target EP, converts if needed
  • `wmk build` targeting CPU with QDQ-quantized model succeeds (auto-converts)
  • `wmk build` targeting NPU with QLinear-quantized model succeeds (auto-converts)
  • No conversion when format already matches EP
  • Compatibility matrix documented and maintained

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.