deepinsight / deepinsight/insightface

arcface_torch: MobileFaceNet batch inference in ONNX

Open
#2,406 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.7k
Forks
6.1k
PR merge metrics
No merged PRs in 30d

Description

Hello.

Currently MobileFaceNet model in arcface_torch throws error when running batch inference after converting to ONNX (pytorch batch inference is fine)

Evironment:

```
- torch: 1.12.1+cu116
- onnx: 1.14.0
- onnxruntime: 1.15.1
```

Reproduce (No training needed):

1. Init MobileFaceNet backbone
```python
from backbones import get_model
backbone_onnx = get_model('mbf', dropout=0.0, fp16=False, num_features=512)
backbone_onnx.eval()
```
2. Convert backbone to ONNX using `convert_onnx` function in `torch2onnx.py`
```python
import numpy as np
import onnx
import torch

def convert_onnx(net, output, opset=11, simplify=False):
assert isinstance(net, torch.nn.Module)
img = np.random.randint(0, 255, size=(112, 112, 3), dtype=np.int32)
img = img.astype(np.float)
img = (img / 255. - 0.5) / 0.5 # torch style norm
img = img.transpose((2, 0, 1))
img = torch.from_numpy(img).unsqueeze(0).float()

torch.onnx.export(net, img, output, input_names=["data"], keep_initializers_as_inputs=False, verbose=False, opset_version=opset)
model = onnx.load(output)
graph = model.graph
graph.input[0].type.tensor_type.shape.dim[0].dim_param = 'None'
if simplify:
from onnxsim import simplify
model, check = simplify(model)
assert check, "Simplified ONNX model could not be validated"
onnx.save(model, output)

output = "mbf.onnx"
convert_onnx(backbone_onnx, output)
```
3. Run `onnx_helper.py` to check ONNX model
Output:
```python
use onnx-model: /content/insightface/recognition/arcface_torch/mbf.onnx
input-shape: ['None', 3, 112, 112]
0 Identity_0
1 Identity_1
2 Identity_2
3 Identity_3
4 Identity_4
5 Identity_5
6 Identity_6
7 Identity_7

---------------------------------------------------------------------------

Fail Traceback (most recent call last)

[](https://localhost:8080/#) in ()
1 handler = ArcFaceORT('/content/insightface/recognition/arcface_torch', cpu=True)
----> 2 err = handler.check()

2 frames

[](https://localhost:8080/#) in check(self, track, test_img)
162 test_img = cv2.resize(test_img, self.image_size)
163 feat, cost = self.benchmark(test_img)
--> 164 batch_result = self.check_batch(test_img)
165 batch_result_sum = float(np.sum(batch_result))
166 if batch_result_sum in [float('inf'), -float('inf')] or batch_result_sum != batch_result_sum:

[](https://localhost:8080/#) in check_batch(self, img)
196 images=imgs, scalefactor=1.0 / self.input_std, size=self.image_size,
197 mean=(self.input_mean, self.input_mean, self.input_mean), swapRB=True)
--> 198 net_out = self.session.run(self.output_names, {self.input_name: blob})[0]
199 return net_out
200

[/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in run(self, output_names, input_feed, run_options)
198 output_names = [output.name for output in self._outputs_meta]
199 try:
--> 200 return self._sess.run(output_names, input_feed, run_options)
201 except C.EPFail as err:
202 if self._enable_fallback:

Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running MatMul node. Name:'MatMul_172' Status Message: matmul_helper.h:61 Compute MatMul dimension mismatch
```

Please take a look, thank you.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.