Convert a quantized conv3d model failed
@zerollzeng is already working on this.
Since Jul 6, 2022.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I'm using pytorch quantization toolkit to quantize my model, which has some conv3d module. The QAT procedure is OK. But when i use trtexec to convert the onnx with Q/DQ pairs to engine, it failed! I have written some simple code to reproduce the error.
Environment
TensorRT Version: 8.2.5.1
NVIDIA GPU: TITAN RTX
NVIDIA Driver Version: 470.103.01
CUDA Version: 11.4
CUDNN Version:
Operating System: ubuntu 20.04
Python Version (if applicable): 3.8.10
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.9.1+cu102
Baremetal or Container (if so, version):
Relevant Files
import torch
import torch.nn as nn
from pytorch_quantization import quant_modules
import pytorch_quantization.nn as quant_nn
quant_modules.initialize()
model = nn.Conv3d(3, 16, kernel_size=(3, 3, 3), stride=(1, 1,1), padding=(1, 1, 1))
quant_nn.TensorQuantizer.use_fb_fake_quant = True
torch.onnx.export(model, torch.randn(1, 3, 20, 20, 20), 'conv3d_quant.onnx', opset_version=13)
Steps To Reproduce
use above code to generate a conv3d onnx with q/dq pair, it should look like this.

Then use
trtexec --onnx=conv3d_quant.onnx --int8 --noTF32 --saveEngine=conv3d_quant.engine
it will throw a error

I guess tensorrt doesn't support conv3d onnx with q/dq pair?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.