NSLocalizedDescription = "Error in declaring network."
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
- I try to do a gesture recognition model using TF, custom tf function.
- The custom functions are needed for data preprocessing and is done in TF because it is easier to communicate with the IOS dev team
- I successfully compressed model as a ml package.
But when I try to do inference I get an error.
- It works when converting to TF Lite
## Stack Trace
```bash
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[77], line 6
4 # Load the model
5 model = ct.models.MLModel(coreml_preprocess_path, compute_units=ct.ComputeUnit.CPU_ONLY)
----> 6 predictions = model.predict(input)
File ~/Documents/XXX/XXX/XXX/XXX/lib/python3.10/site-packages/coremltools/models/model.py:579, in MLModel.predict(self, data)
576 MLModel._check_predict_data(data)
578 if self.__proxy__:
--> 579 return MLModel._get_predictions(self.__proxy__, verify_and_convert_input_dict, data)
580 else: # Error case
581 if _macos_version() < (10, 13):
File ~/Documents/XXX/XXX/XXX/XXX/lib/python3.10/site-packages/coremltools/models/model.py:631, in MLModel._get_predictions(proxy, preprocess_method, data)
629 if type(data) == dict:
630 preprocess_method(data)
--> 631 return proxy.predict(data)
632 else:
633 assert type(data) == list
RuntimeError: {
NSLocalizedDescription = "Error in declaring network.";
}
```
## To Reproduce
- Please add a minimal code example that can reproduce the error when running it.
**OK**
```python
import os
import math
import numpy as np
import tensorflow as tf
import coremltools as ct
from importlib.metadata import version
@tf.function
def tf_rotate_vertically(landmarks):
# Compute vertical line
y = (landmarks[:, 9, 0] + landmarks[:, 13, 0]) / 2
x = (landmarks[:, 9, 1] + landmarks[:, 13, 1]) / 2
# Angle to substract
angle = tf.where(tf.less(x, 0.0), tf.atan(y / x) + np.pi, tf.atan(y / (x + 1e-12)))
angle = tf.where(tf.less(y, 0.0), angle - (2.0 * np.pi), angle)
angle = tf.where(tf.logical_and(tf.equal(x, 0.0), tf.equal(y, 0.0)), tf.zeros_like(x), angle)
# Get rotation angle radian to have vertical hand
angle_radian = float(math.pi / 2) - angle
# Create the rotation matric
rotation_matrix = tf.stack(
[
(tf.cos(angle_radian), -tf.sin(angle_radian)),
(tf.sin(angle_radian), tf.cos(angle_radian))
],
axis=0
)
rotation_matrix = tf.transpose(rotation_matrix)
landmarks_rotated = tf.linalg.matmul(landmarks, rotation_matrix)
return landmarks_rotated
@tf.function
def tf_atan2(y, x):
angle = tf.where(tf.less(x, 0.0), tf.atan(y / x) + np.pi, tf.atan(y / (x + 1e-12)))
angle = tf.where(tf.less(y, 0.0), angle - (2.0 * np.pi), angle)
angle = tf.where(tf.logical_and(tf.equal(x, 0.0), tf.equal(y, 0.0)), tf.zeros_like(x), angle)
return angle
# Keras model
# Model inputs
input_scalar = tf.keras.layers.Input(shape=(1), dtype=tf.float32, name='handedness')
input_matrix = tf.keras.layers.Input(shape=(21, 2), dtype=tf.float32, name='landmarks')
# Keep first idx of each sample
first_element = tf.gather(input_matrix, 0, axis=1, name='first_element')
# Substract by first element
x = tf.keras.layers.subtract([input_matrix, first_element], name='substract', input_shape=(21, 2))
# Rotation
x = tf.keras.layers.Lambda(tf_rotate_vertically, name='rotate_vertically', input_shape=(21, 2))(x)
preprocessing_model = tf.keras.Model(inputs=input_matrix, outputs=x)
preprocessing_model.predict(tf.reshape(np.random.rand(10, 21, 2), [10, 21, 2])) # OK
# Save model
# Convert the model to Core ML
input_matrix_shape = ct.Shape(shape=(1, 21, 2))
coreml_model = ct.convert(
preprocessing_model,
inputs=[
ct.TensorType(name='landmarks', shape=input_matrix_shape),
],
source='tensorflow',
convert_to="mlprogram"
)
coreml_preprocess_path = '../model/IOS/preprocess_model.mlpackage'
# Save the model
coreml_model.save(coreml_preprocess_path)
```
**KO**
```python
# Random Input to test loaded ml
input = {
"landmarks": np.random.rand(1, 21, 2),
}
# Load the model
model = ct.models.MLModel(coreml_preprocess_path, compute_units=ct.ComputeUnit.CPU_ONLY)
predictions = model.predict(input)
```
## System environment (please complete the following information):
- coremltools version: 7.0
- numpy version: 1.24.3
- tensorflow version: 2.13.0
- OS (e.g. MacOS version or Linux type): M1 Pro 13.6
Contributor guide
Research direction
The issue names no repository files or tests; start by running the provided TensorFlow conversion and MLModel.predict reproduction with coremltools 7.0, TensorFlow 2.13.0, and NumPy 1.24.3. Trace where the loaded mlpackage raises “Error in declaring network”; done means identifying the reproducible cause and whether prediction can work for this model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100