deepinsight / deepinsight/insightface
arcface_torch: MobileFaceNet batch inference in ONNX
- Dominant language
- Python
- Stars
- 29.7k
- Forks
- 6.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hello.
Currently MobileFaceNet model in arcface_torch throws error when running batch inference after converting to ONNX (pytorch batch inference is fine)
Evironment:
```
- torch: 1.12.1+cu116
- onnx: 1.14.0
- onnxruntime: 1.15.1
```
Reproduce (No training needed):
1. Init MobileFaceNet backbone
```python
from backbones import get_model
backbone_onnx = get_model('mbf', dropout=0.0, fp16=False, num_features=512)
backbone_onnx.eval()
```
2. Convert backbone to ONNX using `convert_onnx` function in `torch2onnx.py`
```python
import numpy as np
import onnx
import torch
def convert_onnx(net, output, opset=11, simplify=False):
assert isinstance(net, torch.nn.Module)
img = np.random.randint(0, 255, size=(112, 112, 3), dtype=np.int32)
img = img.astype(np.float)
img = (img / 255. - 0.5) / 0.5 # torch style norm
img = img.transpose((2, 0, 1))
img = torch.from_numpy(img).unsqueeze(0).float()
torch.onnx.export(net, img, output, input_names=["data"], keep_initializers_as_inputs=False, verbose=False, opset_version=opset)
model = onnx.load(output)
graph = model.graph
graph.input[0].type.tensor_type.shape.dim[0].dim_param = 'None'
if simplify:
from onnxsim import simplify
model, check = simplify(model)
assert check, "Simplified ONNX model could not be validated"
onnx.save(model, output)
output = "mbf.onnx"
convert_onnx(backbone_onnx, output)
```
3. Run `onnx_helper.py` to check ONNX model
Output:
```python
use onnx-model: /content/insightface/recognition/arcface_torch/mbf.onnx
input-shape: ['None', 3, 112, 112]
0 Identity_0
1 Identity_1
2 Identity_2
3 Identity_3
4 Identity_4
5 Identity_5
6 Identity_6
7 Identity_7
---------------------------------------------------------------------------
Fail Traceback (most recent call last)
[](https://localhost:8080/#) in ()
1 handler = ArcFaceORT('/content/insightface/recognition/arcface_torch', cpu=True)
----> 2 err = handler.check()
2 frames
[](https://localhost:8080/#) in check(self, track, test_img)
162 test_img = cv2.resize(test_img, self.image_size)
163 feat, cost = self.benchmark(test_img)
--> 164 batch_result = self.check_batch(test_img)
165 batch_result_sum = float(np.sum(batch_result))
166 if batch_result_sum in [float('inf'), -float('inf')] or batch_result_sum != batch_result_sum:
[](https://localhost:8080/#) in check_batch(self, img)
196 images=imgs, scalefactor=1.0 / self.input_std, size=self.image_size,
197 mean=(self.input_mean, self.input_mean, self.input_mean), swapRB=True)
--> 198 net_out = self.session.run(self.output_names, {self.input_name: blob})[0]
199 return net_out
200
[/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in run(self, output_names, input_feed, run_options)
198 output_names = [output.name for output in self._outputs_meta]
199 try:
--> 200 return self._sess.run(output_names, input_feed, run_options)
201 except C.EPFail as err:
202 if self._enable_fallback:
Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running MatMul node. Name:'MatMul_172' Status Message: matmul_helper.h:61 Compute MatMul dimension mismatch
```
Please take a look, thank you.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.