apple / apple/coremltools

Conversion of Tensorflow 1.x object detection model to coreml using coremltools

Open
#1,138 1 comment 0 reactions 0 assignees View on GitHub
bug tf1.x
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describe the bug
Hello ,
I have trained object detection model using tensorflow object detection api 1. I have exported the model in .pb format . I want to use it in the ios so i tried converting it to .mlmodel . But i am getting some errors like 'Input max_boxes must be const at compile time'. I have given details below.
I am using SSD-MobileNetV1 model

## Trace
Running TensorFlow Graph Passes: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 7/7 [00:01<00:00, 4.36 passes/s]
Converting Frontend ==> MIL Ops: 45%|β–ˆβ–ˆβ–ˆβ–ˆβ– | 323/719 [00:00<00:00, 3226.29 ops/s]
Converting Frontend ==> MIL Ops: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 9/9 [00:00<00:00, 4198.97 ops/s]

Converting Frontend ==> MIL Ops: 0%| | 0/30 [00:00.tensor'>
WARNING:root:Input ls elem type unknown. Override with .tensor'>
Converting Frontend ==> MIL Ops: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 30/30 [00:00<00:00, 1523.76 ops/s]

Converting Frontend ==> MIL Ops: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 9/9 [00:00<00:00, 6113.16 ops/s]

Converting Frontend ==> MIL Ops: 0%| | 0/30 [00:00.tensor'>
WARNING:root:Input ls elem type unknown. Override with .tensor'>
Converting Frontend ==> MIL Ops: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 30/30 [00:00<00:00, 1438.46 ops/s]
Converting Frontend ==> MIL Ops: 93%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž| 666/719 [00:01<00:00, 294.71 ops/s]
Converting Frontend ==> MIL Ops: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 9/9 [00:00<00:00, 1378.75 ops/s]

Converting Frontend ==> MIL Ops: 0%| | 0/381 [00:00 MIL Ops: 28%|β–ˆβ–ˆβ–Š | 108/381 [00:00<00:00, 1076.34 ops/s]
Converting Frontend ==> MIL Ops: 47%|β–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 178/381 [00:00<00:00, 507.98 ops/s]
Converting Frontend ==> MIL Ops: 93%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž| 666/719 [00:01<00:00, 351.75 ops/s]
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
in ()
2 model = "inference_graph/frozen_inference_graph.pb"
3 outputs=['detection_classes:0','detection_scores:0','detection_boxes:0','num_detections:0']
----> 4 coremltools.convert(model,outputs=outputs)

22 frames
/usr/local/lib/python3.7/dist-packages/coremltools/converters/mil/mil/input_type.py in validate_inputs(self, op_name, op_type, candidate_kvs)
112 msg = msg_prefix + \
113 'Input {} must be const at compile time'
--> 114 raise ValueError(msg.format(name), name, var.name)
115
116 if not isinstance(var, InternalVar) and \

ValueError: ('Op "non_maximum_suppression_8" (op_type: non_maximum_suppression) Input max_boxes must be const at compile time', 'max_boxes', 'Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Minimum')

## To Reproduce
import coremltools
model = "inference_graph/frozen_inference_graph.pb"
outputs=['detection_classes:0','detection_scores:0','detection_boxes:0','num_detections:0']
coremltools.convert(model,outputs=outputs)
# Paste code snippet here
```
- If applicable, please attach the source model
please find source model below :
[https://drive.google.com/file/d/114dq7RPQrDi-PX2qzREuPPj9ilblm4z3/view?usp=sharing](url)

## System environment (please complete the following information):
- coremltools version (e.g., 3.0b5): ](url) 4.1
- OS (e.g., MacOS, Linux): Windows
- macOS version (if applicable): NA
- XCode version (if applicable): NA
- How you install python (anaconda, virtualenv, system):sytem
- python version (e.g. 3.7): 3.7
- any other relevant information: tensorflow version 1.15

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the linked frozen TensorFlow model, the shown coremltools.convert call, TensorFlow 1.15, and coremltools 4.1. Trace the non_maximum_suppression error for the max_boxes input and determine whether this model can be converted. Done means the conversion succeeds for this reproduction or the limitation is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.