NVIDIA / NVIDIA/TensorRT

Convert a quantized conv3d model failed

Open
#2,125 13 comments 0 reactions 1 assignee View on GitHub

@zerollzeng is already working on this.

Since Jul 6, 2022.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

I'm using pytorch quantization toolkit to quantize my model, which has some conv3d module. The QAT procedure is OK. But when i use trtexec to convert the onnx with Q/DQ pairs to engine, it failed! I have written some simple code to reproduce the error.

Environment

TensorRT Version: 8.2.5.1
NVIDIA GPU: TITAN RTX
NVIDIA Driver Version: 470.103.01
CUDA Version: 11.4
CUDNN Version:
Operating System: ubuntu 20.04
Python Version (if applicable): 3.8.10
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.9.1+cu102
Baremetal or Container (if so, version):

Relevant Files

import torch
import torch.nn as nn
from pytorch_quantization import quant_modules
import pytorch_quantization.nn as quant_nn
quant_modules.initialize()

model = nn.Conv3d(3, 16, kernel_size=(3, 3, 3), stride=(1, 1,1), padding=(1, 1, 1))
quant_nn.TensorQuantizer.use_fb_fake_quant = True
torch.onnx.export(model, torch.randn(1, 3, 20, 20, 20), 'conv3d_quant.onnx', opset_version=13)

Steps To Reproduce

use above code to generate a conv3d onnx with q/dq pair, it should look like this.
2022-07-06 20-23-08 的屏幕截图

Then use

trtexec --onnx=conv3d_quant.onnx --int8 --noTF32 --saveEngine=conv3d_quant.engine

it will throw a error
2022-07-06 20-25-50 的屏幕截图

I guess tensorrt doesn't support conv3d onnx with q/dq pair?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.