apple / apple/coremltools

unsupported for inputs/outputs of the model

Open
#1,744 3 comments 0 reactions 0 assignees View on GitHub
bug tf2.x / tf.keras
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
Hello,

I tried to convert the EfficientDet Lite2 model found on tensorflowhub [here](https://tfhub.dev/tensorflow/efficientdet/lite2/detection/1) using the saved_model directory. So I used simple coremltools `convert()` but it crashes when **Running TensorFlow Graph Passes**

## Stack Trace
```Running TensorFlow Graph Passes: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:01<00:00, 3.46 passes/s]
Traceback (most recent call last):
File "/Users/smurf/Desktop/repositories/playground/convert_coreml.py", line 10, in
mlmodel = ct.convert("efficientdet_lite2_detection_1")
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/_converters_entry.py", line 444, in convert
mlmodel = mil_convert(
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 190, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 217, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 282, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/converter.py", line 102, in __call__
return tf2_loader.load()
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 82, in load
program = self._program_from_tf_ssa()
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/frontend/tensorflow2/load.py", line 204, in _program_from_tf_ssa
converter = TF2Converter(
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/frontend/tensorflow2/converter.py", line 16, in __init__
TFConverter.__init__(self, tf_ssa, inputs, outputs, opset_version)
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 196, in __init__
inputs.append(TensorType(name=inp, shape=shape, dtype=dtype))
File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/input_types.py", line 215, in __init__
raise TypeError("dtype={} is unsupported for inputs/outputs of the model".format(dtype))
TypeError: dtype=.int'> is unsupported for inputs/outputs of the model
```

## To Reproduce
```
import tensorflow as tf
import coremltools as ct
import numpy as np

# imported = tf.saved_model.load("efficientdet_lite2_detection_1")
# model = imported.signatures["serving_default"]
# image = np.ones((1, 448, 448, 3), dtype = np.uint8)
# output = model(images=image) ---> {output_0 : shape=(1, 100, 4), dtype=float32, output_1: shape=(1, 100), dtype=float32, output_2: shape=(1, 100), dtype=float32, output_3: shape=(1,), dtype=int32}
mlmodel = ct.convert("efficientdet_lite2_detection_1")
```

## System environment (please complete the following information):
- MacOS Monterey 12.6 M1:

```
$ conda list

# Name Version Build Channel
absl-py 1.4.0 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
ca-certificates 2023.01.10 hca03da5_0
cachetools 5.3.0 pypi_0 pypi
certifi 2022.12.7 py39hca03da5_0
charset-normalizer 3.0.1 pypi_0 pypi
coremltools 6.1 pypi_0 pypi
flatbuffers 23.1.21 pypi_0 pypi
gast 0.4.0 pypi_0 pypi
google-auth 2.16.0 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
grpcio 1.51.1 pypi_0 pypi
h5py 3.8.0 pypi_0 pypi
idna 3.4 pypi_0 pypi
importlib-metadata 6.0.0 pypi_0 pypi
keras 2.8.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
libclang 15.0.6.1 pypi_0 pypi
libcxx 14.0.6 h848a8c0_0
libffi 3.4.2 hca03da5_6
markdown 3.4.1 pypi_0 pypi
markupsafe 2.1.2 pypi_0 pypi
mpmath 1.2.1 pypi_0 pypi
ncurses 6.3 h1a28f6b_3
numpy 1.23.1 pypi_0 pypi
oauthlib 3.2.2 pypi_0 pypi
openssl 1.1.1s h1a28f6b_0
opt-einsum 3.3.0 pypi_0 pypi
packaging 23.0 pypi_0 pypi
pip 22.3.1 py39hca03da5_0
protobuf 3.19.6 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
python 3.9.16 hc0d8a6c_0
readline 8.2 h1a28f6b_0
requests 2.28.2 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
setuptools 65.6.3 py39hca03da5_0
six 1.16.0 pypi_0 pypi
sqlite 3.40.1 h7a7dc30_0
sympy 1.11.1 pypi_0 pypi
tensorboard 2.8.0 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorflow-estimator 2.10.0 pypi_0 pypi
tensorflow-macos 2.8.0 pypi_0 pypi
termcolor 2.2.0 pypi_0 pypi
tf-estimator-nightly 2.8.0.dev2021122109 pypi_0 pypi
tk 8.6.12 hb8d0fd4_0
tqdm 4.64.1 pypi_0 pypi
typing-extensions 4.4.0 pypi_0 pypi
tzdata 2022g h04d1e81_0
urllib3 1.26.14 pypi_0 pypi
werkzeug 2.2.2 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0
wrapt 1.14.1 pypi_0 pypi
xz 5.2.10 h80987f9_1
zipp 3.11.0 pypi_0 pypi
zlib 1.2.13 h5a0b063_0
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the EfficientDet Lite2 SavedModel and the ct.convert() call shown in the issue. Start in converters/mil/frontend/tensorflow2/converter.py and converters/mil/input_types.py, where the traceback rejects the int32 output dtype. Done means conversion no longer fails for the reported model, with coverage for its integer input/output types.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.