openvinotoolkit / openvinotoolkit/nncf

SmoothQuant is constructed before IgnoredScope is applied, so no config can avoid the AttributeError on weight-sharing transformers

Open
#4,172 4 comments 0 reactions 1 assignee View on GitHub

@andrey-churkin is already working on this.

Since Aug 14, 2026.

Dominant language
Python
Stars
1.2k
Forks
305
Avg merge
1d 9h
Merged PRs (30d)
27

Description

Summary

nncf.quantize(..., model_type=ModelType.TRANSFORMER) raises AttributeError inside SmoothQuant's get_weight_value on an RT-DETRv2 ONNX export, and no IgnoredScope a caller supplies can avoid it, because SmoothQuant is constructed before the ignored scope is applied.

Affects: 3.2.0 through develop (3.3.0 does not fix it).

Root cause, from the source

In src/nncf/quantization/algorithms/post_training/pipeline.py, SmoothQuant is appended as the first pipeline step (lines 81-99) whenever model_type == TRANSFORMER and smooth-quant alpha is >= 0 — and it is constructed without ignored_scope. That argument is passed only to the subsequent MinMax step (lines 113-114).

In the OpenVINO backend, get_weight_valueget_weight_tensor_port_idnode.layer_attributes.get_const_port_ids() returns None for a weight shared across decoder layers via Identity nodes — which is exactly how RT-DETRv2 shares its query_pos_head MLP.

Why it is not merely cosmetic

model_type=TRANSFORMER is the single most effective lever we found for transformer detectors: it recovered RT-DETRv2 INT8 from 0.4921 to 0.6680 mAP on host, and to 0.6744 on Tiger Lake. The crash forces every user of a weight-sharing transformer to disable SmoothQuant wholesale to get any of that benefit.

Workaround

AdvancedQuantizationParameters(
    smooth_quant_alphas=AdvancedSmoothQuantParameters(convolution=-1, matmul=-1)
)

This makes the >= 0 guard false and drops SmoothQuant from the pipeline entirely.

Suggested fix

Pass ignored_scope into the SmoothQuant constructor, or defer its construction until after the ignored-scope resolution that MinMax already receives.

Verified

SmoothQuant's openvino_backend.py and algorithm.py are byte-identical (md5) across v3.2.0, v3.3.0 and develop; pipeline.py is byte-identical between 3.2.0 and 3.3.0.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.