microsoft / microsoft/onnxruntime-inference-examples
Do QuantizeLinear and DequantizeLinear operators only run with CPU?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 414
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 14
Description
I followed ONNX Runtime Quantization Example to get quantized model (https://github.com/microsoft/onnxruntime-inference-examples/tree/main/quantization/image_classification/cpu) and got the output model called mobilenetv2-7.quant.onnx. After quantizing the model, I tried to get output result by running inference. However, when I used onnxruntime-gpu and set all session run parameters for GPU (https://onnxruntime.ai/docs/api/python/api_summary.html#inferencesession) to run mobilenetv2-7.quant.onnx, I found that my computer used CPU to run the model instead of GPU. Besides, I used same session run parameters to run FP model called mobilenetv2-7-infer.onnx and my computer used GPU to run the output inference normally. Hence, I wonder to know if QuantizeLinear and DequantizeLinear operators only run with CPU or I can run these operators with GPU by changing my session run parameters, Thanks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ONNX Runtime Quantization Example and the Python InferenceSession API linked in the issue. Compare execution of mobilenetv2-7.quant.onnx with mobilenetv2-7-infer.onnx under the same GPU settings, then establish whether QuantizeLinear and DequantizeLinear support the GPU provider and document the required configuration or limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100