microsoft / microsoft/onnxruntime-inference-examples

Do QuantizeLinear and DequantizeLinear operators only run with CPU?

Open
#166 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.7k
Forks
414
Avg merge
1d 6h
Merged PRs (30d)
14

Description

I followed ONNX Runtime Quantization Example to get quantized model (https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/image_classification/cpu) and got the output model called mobilenetv2-7.quant.onnx. After quantizing the model, I tried to get output result by running inference. However, when I used onnxruntime-gpu and set all session run parameters for GPU (https://onnxruntime.ai/docs/api/python/api_summary.html#inferencesession) to run mobilenetv2-7.quant.onnx, I found that my computer used CPU to run the model instead of GPU. Besides, I used same session run parameters to run FP model called mobilenetv2-7-infer.onnx and my computer used GPU to run the output inference normally. Hence, I wonder to know if QuantizeLinear and DequantizeLinear operators only run with CPU or I can run these operators with GPU by changing my session run parameters, Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ONNX Runtime Quantization Example and the Python InferenceSession API linked in the issue. Compare execution of mobilenetv2-7.quant.onnx with mobilenetv2-7-infer.onnx under the same GPU settings, then establish whether QuantizeLinear and DequantizeLinear support the GPU provider and document the required configuration or limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.