pytorch / pytorch/vision

Faster/Mask RCNN (fasterrcnn_mobilenet_v2_fpn) model export to ONNX. Warnings.

Open
#3,044 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: onnx topic: object detection
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

Hello,

I have a lot of 'UserWarnings' during exporting model to ONNX. Can someone clarify meanings of this warnings and help to eliminate then. I am creating model from this finetuning tutorial and have no problem during training/torch evaluation.
However encounter one 'RuntimeWarning' and about 7 'UserWarning' while exporting trained model to ONNX. There are also several warnings during evaluation ONNX model on onnxruntime.

To Reproduce

import torch
import torchvision
from torchvision.models.detection import FasterRCNN
from torchvision.models.detection.rpn import AnchorGenerator
from torchvision.models.detection.faster_rcnn import FastRCNNPredictor

def create_faster_mobilenet_v2():
    backbone = torchvision.models.mobilenet_v2(pretrained=True).features
    backbone.out_channels = 1280
    anchor_generator = AnchorGenerator(sizes=((32, 64, 128, 256, 512),),
                                       aspect_ratios=((0.5, 1.0, 2.0),))
    num_classes = 47
    roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0'], output_size=7, sampling_ratio=2)
    model = FasterRCNN(backbone,
                       num_classes=num_classes,
                       rpn_anchor_generator=anchor_generator,
                       box_roi_pool=roi_pooler)
    in_features = model.roi_heads.box_predictor.cls_score.in_features
    model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
    return model

# export to ONNX
batch_size = 1
# load model
model = create_faster_mobilenet_v2()

# Dummy input to the model
x = torch.randn(batch_size, 3, 800, 800)

# set the model to inference mode
model.eval()

# running dummy image
torch_out = model(x)

print('Model outputs: ', torch_out[0]['boxes'].shape, torch_out[0]['labels'].shape, torch_out[0]['scores'].shape)
print('Model output boxes: ', torch_out[0]['boxes'])
print('Model output labels: ', torch_out[0]['labels'])
print('Model output scores: ', torch_out[0]['scores'])

# Export the model
torch.onnx.export(model,  # model being run
                  x,  # model input (or a tuple for multiple inputs)
                  "fasterRCNN-231120-694-46C-batch1.onnx",  # model path + name
                  # export_params=True,  # store the trained parameter weights inside the model file
                  opset_version=11,  # the ONNX version to export the model to
                  # do_constant_folding=True,  # whether to execute constant folding for optimization
                  input_names=['input'],  # the model's input names
                  output_names=['output'],  # the model's output names
                  dynamic_axes={'input': {0: 'batch_size'},  # variable length axes
                                            'output': {0: 'batch_size'}
                                }
                  )

Output during export:

Model outputs:  torch.Size([3, 4]) torch.Size([3]) torch.Size([3])

Model output boxes:  tensor([[2.6542e+00, 2.4133e-01, 1.1089e+02, 8.9261e+01],
        [2.1360e+00, 6.8483e+02, 1.0292e+02, 7.9821e+02],
        [9.6579e-01, 7.3022e+02, 1.3740e+02, 7.9860e+02]],
       grad_fn=<StackBackward>)

Model output labels:  tensor([3, 1, 1])

Model output scores:  tensor([0.0778, 0.0638, 0.0519], grad_fn=<IndexBackward>)

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torch\nn\functional.py:3123: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requ
ires_grad_(True), rather than torch.tensor(sourceTensor).
  dtype=torch.float32)).float())) for i in range(dim)]

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torchvision\models\detection\anchor_utils.py:147: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.
clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(image_size[1] // g[1], dtype=torch.int64, device=device)] for g in grid_sizes]

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torch\tensor.py:593: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iter
ations executed (and might lead to errors or silently give incorrect results).
  'incorrect results).', category=RuntimeWarning)

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torchvision\ops\boxes.py:128: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().req
uires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torchvision\ops\boxes.py:130: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().req
uires_grad_(True), rather than torch.tensor(sourceTensor).

  boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))
D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torchvision\models\detection\transform.py:271: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clo
ne().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  for s, s_orig in zip(new_size, original_size)

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torch\onnx\symbolic_opset9.py:2378: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators,
 including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  "If indices include negative values, the exported graph will produce incorrect results.")

D:\Projects\MachineLearning\fasterRCNN\train\venv\lib\site-packages\torch\onnx\symbolic_opset9.py:588: UserWarning: This model contains a squeeze operation on dimension 1 on an input with unknown shape. Note that if the size of dimensi
on 1 of the input is not 1, the ONNX model will return an error. Opset version 11 supports squeezing on non-singleton dimensions, it is recommended to export this model using opset version 11 or higher.
  "version 11 or higher.")

Environment

  1. PyTorch Version : 1.7.0
  2. Torchvision: 0.8.1
  3. OS : Wndows 10
  4. How you installed PyTorch (conda, pip, source): pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
  5. Python version: 3.6.5
  6. CUDA/cuDNN version: 10.2 / 11.0 (I have another machine - result is the same)
  7. onnx: 1.7.0
  8. onnxruntime: 1.5.2

With respect.
Bulat.

cc @neginraoof

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the export with PyTorch 1.7.0, torchvision 0.8.1, ONNX 1.7.0, and onnxruntime 1.5.2 using the supplied Faster R-CNN script. Start by reviewing the warning locations in torch/nn/functional.py, torchvision/models/detection/anchor_utils.py, torchvision/ops/boxes.py, torchvision/models/detection/transform.py, and torch/onnx/symbolic_opset9.py. Done should include an explained or eliminated warning set and successful ONNX Runtime evaluation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.