apple / apple/coremltools

Support TFLite_Detection_PostProcess Op

Open
#642 3 comments 9 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describe the bug
I am trying to convert tensorflow SSD MobilenetV2 model (frozen graph) to coreml model using
[coremltools.converters.tensorflow.convert function](https://apple.github.io/coremltools/generated/coremltools.converters.tensorflow.convert.html) and getting error `Op type not registered 'TFLite_Detection_PostProcess' in binary `.

As far as I understand, this Op is not supported in CoreML. Here is its C++ implementation: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/detection_postprocess.cc

This blocks me from using CoreML for my detection problem. Any thoughts on:
1. When the Op will be supported?
2. Is there a quick workaround? Implementing and maintaining that code in Swift is too hard

## Trace
```
Traceback (most recent call last):
File "coreml.py", line 144, in
main()
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "coreml.py", line 129, in main
mlmodel_path
File "coreml.py", line 44, in tf_convert_to_coreml
the_graph = load_graph(tf_model_path)
File "coreml.py", line 62, in load_graph
tfc.import_graph_def(original_gdef, name='')
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/Users/aminbenarieb/.virtualenvs/coreml_1.14/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 427, in import_graph_def
graph._c_graph, serialized, options) # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
```'TFLite_Detection_PostProcess' in binary running on Amins-MBP.Dlink. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.
```

## My Code
```
import coremltools

def convert_to_coreml(
model_path,
prototxt_path,
class_labels_path,
mlmodel_path
):
print(model_path, prototxt_path)
coreml_model = coremltools.converters.tensorflow.convert(
model_path, predicted_feature_name=class_labels_path
)

coreml_model.save(mlmodel_path)
```

## System environment:
- coreml version: 3.3
- tensorflow versions: 2.1 (used for TF frozen graph):
- macOS version 10.15):
- How you install python (virtualenv):
- python 3.7.6:

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with the supplied coremltools.converters.tensorflow.convert call and the TensorFlow frozen graph containing TFLite_Detection_PostProcess. Read TensorFlow Lite's linked detection_postprocess.cc implementation and trace how TensorFlow operations are registered and converted. Done means the SSD MobileNetV2 graph converts successfully without the unregistered-op error, with the resulting model behavior checked.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.