[Feature]: Expose SmoothQuant alpha for int8_sq quantization in quantize.py
@laikhtewari is already working on this.
Since Jul 8, 2026.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
🚀 The feature, motivation and pitch
ModelOpt's SmoothQuant calibration defaults to alpha=1.0 and TensorRT-LLM's
examples/quantization/quantize.py exposes no way to change it — the only override in the
tree is the hardcoded Gemma special case in quantize_by_modelopt.py
({"method": "smoothquant", "alpha": 0.5}). The SmoothQuant paper recommends 0.5–0.9
depending on the model.
In our sweeps on Qwen3-0.6B/1.7B (11 alphas × 2 calib sets, real TRT engines, GSM8K-1319),
alpha=1.0 was never optimal and cost up to 20 GSM8K points vs the best alpha in the same
setting (worst case: Qwen3-0.6B, 15.39% at alpha=1.0 vs 35.41% at alpha=0.85, against a
36.32% bf16 baseline).
Proposal: add --smoothquant_alpha (float, default None = current behavior) to quantize.py
and thread it to quantize_and_export, where it sets the same algorithm dict the Gemma
special case already uses. Backward compatible; ModelOpt already pydantic-validates the
field (bounds [0, 1]). Happy to submit the PR (change is ready).
Alternatives
ModelOpt's hf_ptq.py has a --recipe mechanism, so an alternative is to contribute a recipe
YAML on the Model-Optimizer side instead. But TRT-LLM's quantize.py has no recipe mechanism,
so exposing the flag directly is the minimal, self-contained fix for this CLI. The two
approaches don't conflict.
Additional context
Full sweep (11 alphas × 2 calib sets × 2 Qwen3 sizes):
| Model | Calib | alpha=1.0 (current) | best alpha | GSM8K cost of default |
|---|---|---|---|---|
| Qwen3-1.7B | wikitext | 39.80% | 57.09% @ α=0.75 | −17.3 pts |
| Qwen3-1.7B | gsm8k-train | 61.87% | 63.76% @ α=0.85 | −1.9 pts |
| Qwen3-0.6B | wikitext | 21.76% | 31.54% @ α=0.75 | −9.8 pts |
| Qwen3-0.6B | gsm8k-train | 15.39% | 35.41% @ α=0.85 | −20.0 pts |
The PR is ready: +121/−1, includes a GPU-free unit test
(tests/unittest/trt/quantization/test_smoothquant_alpha.py). No behavior change when the
flag is unset.
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.