tensorflow / tensorflow/models

CenterNet MobileNetV2 - inference is too slow

Open
#10,006 12 comments 2 reactions 3 assignees View on GitHub

@jch1 is already working on this.

Since May 21, 2021.

models:research:odapi type:docs
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am able to run SSD MobileNetV2 and CenterNet MobileNetV2 (boxes prediction) on my android device. When I compare inference speed of the models on my android device I get below results:

inference of CenterNet MobileNetV2 512x512 is aprox. 5-6 times slower than SSD MobileNetV2 320x320 quantized model.
inference of CenterNet MobileNetV2 512x512 is aprox. 3 times slower than SSD MobileNetV2 320x320 non-quantized model.
inference of CenterNet MobileNetV2 512x512 has aprox. same speed as SSD MobileNetV2 640x640 quantized model.

I run the models on CPU only, using 4 threads based on the official android tflite tutorial (6 month old, hopefully it is not the reason). I have used CenterNet MobileNetV2 512x512 tflite model directly from here: http://download.tensorflow.org/models/object_detection/tf2/20210210/centernet_mobilenetv2fpn_512x512_coco17_kpts.tar.gz (which has btw input size 320x320 instead of 512x512) and I used also CenterNet MobileNetV2 512x512 tflite model created based on this tutorial: https://github.com/tensorflow/models/blob/master/research/object_detection/colab_tutorials/centernet_on_device.ipynb
both CenterNet MobileNetV2 512x512 tflite models have the same inference speed.

My understanding is the inference of CenterNet MobileNetV2 512x512 should be aprox. 3 times faster than SSD MobileNetV2 320x320 (based on official documentation/benchmark below)

Model name Speed (ms) COCO mAP Outputs
CenterNet MobileNetV2 FPN 512x512 6 23.4 Boxes
SSD MobileNet v2 320x320 19 20.2 Boxes
SSD MobileNet V1 FPN 640x640 48 29.1 Boxes
SSD MobileNet V2 FPNLite 320x320 22 22.2 Boxes
SSD MobileNet V2 FPNLite 640x640 39 28.2 Boxes

What is your experience with inference speed of CenterNet MobileNetV2 512x512? Is it fast for you? Am I missing something? Why is CenterNet MobileNetV2 512x512 so slow for me?

thank you

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.