Support TFLite_Detection_PostProcess Op
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describe the bug
I am trying to convert tensorflow SSD MobilenetV2 model (frozen graph) to coreml model using
[coremltools.converters.tensorflow.convert function](https://apple.github.io/coremltools/generated/coremltools.converters.tensorflow.convert.html) and getting error `Op type not registered 'TFLite_Detection_PostProcess' in binary `.
As far as I understand, this Op is not supported in CoreML. Here is its C++ implementation: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/detection_postprocess.cc
This blocks me from using CoreML for my detection problem. Any thoughts on:
1. When the Op will be supported?
2. Is there a quick workaround? Implementing and maintaining that code in Swift is too hard
## Trace
```
Traceback (most recent call last):
File "coreml.py", line 144, in
main()
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "coreml.py", line 129, in main
mlmodel_path
File "coreml.py", line 44, in tf_convert_to_coreml
the_graph = load_graph(tf_model_path)
File "coreml.py", line 62, in load_graph
tfc.import_graph_def(original_gdef, name='')
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 427, in import_graph_def
graph._c_graph, serialized, options) # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
```'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
```
## My Code
```
import coremltools
def convert_to_coreml(
model_path,
prototxt_path,
class_labels_path,
mlmodel_path
):
print(model_path, prototxt_path)
coreml_model = coremltools.converters.tensorflow.convert(
model_path, predicted_feature_name=class_labels_path
)
coreml_model.save(mlmodel_path)
```
## System environment:
- coreml version: 3.3
- tensorflow versions: 2.1 (used for TF frozen graph):
- macOS version 10.15):
- How you install python (virtualenv):
- python 3.7.6:
Contributor guide
Research direction
Start by reproducing the failure with the supplied coremltools.converters.tensorflow.convert call and the TensorFlow frozen graph containing TFLite_Detection_PostProcess. Read TensorFlow Lite's linked detection_postprocess.cc implementation and trace how TensorFlow operations are registered and converted. Done means the SSD MobileNetV2 graph converts successfully without the unregistered-op error, with the resulting model behavior checked.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100