apple / apple/coremltools

NSLocalizedDescription = "Error in declaring network."

Open
#2,019 6 comments 4 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
- I try to do a gesture recognition model using TF, custom tf function.
- The custom functions are needed for data preprocessing and is done in TF because it is easier to communicate with the IOS dev team
- I successfully compressed model as a ml package.
But when I try to do inference I get an error.
- It works when converting to TF Lite

## Stack Trace
```bash
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[77], line 6
4 # Load the model
5 model = ct.models.MLModel(coreml_preprocess_path, compute_units=ct.ComputeUnit.CPU_ONLY)
----> 6 predictions = model.predict(input)

File ~/Documents/XXX/XXX/XXX/XXX/lib/python3.10/site-packages/coremltools/models/model.py:579, in MLModel.predict(self, data)
576 MLModel._check_predict_data(data)
578 if self.__proxy__:
--> 579 return MLModel._get_predictions(self.__proxy__, verify_and_convert_input_dict, data)
580 else: # Error case
581 if _macos_version() < (10, 13):

File ~/Documents/XXX/XXX/XXX/XXX/lib/python3.10/site-packages/coremltools/models/model.py:631, in MLModel._get_predictions(proxy, preprocess_method, data)
629 if type(data) == dict:
630 preprocess_method(data)
--> 631 return proxy.predict(data)
632 else:
633 assert type(data) == list

RuntimeError: {
NSLocalizedDescription = "Error in declaring network.";
}

```

## To Reproduce
- Please add a minimal code example that can reproduce the error when running it.
**OK**
```python
import os
import math
import numpy as np
import tensorflow as tf
import coremltools as ct
from importlib.metadata import version

@tf.function
def tf_rotate_vertically(landmarks):
# Compute vertical line
y = (landmarks[:, 9, 0] + landmarks[:, 13, 0]) / 2
x = (landmarks[:, 9, 1] + landmarks[:, 13, 1]) / 2

# Angle to substract
angle = tf.where(tf.less(x, 0.0), tf.atan(y / x) + np.pi, tf.atan(y / (x + 1e-12)))
angle = tf.where(tf.less(y, 0.0), angle - (2.0 * np.pi), angle)
angle = tf.where(tf.logical_and(tf.equal(x, 0.0), tf.equal(y, 0.0)), tf.zeros_like(x), angle)

# Get rotation angle radian to have vertical hand
angle_radian = float(math.pi / 2) - angle

# Create the rotation matric
rotation_matrix = tf.stack(
[
(tf.cos(angle_radian), -tf.sin(angle_radian)),
(tf.sin(angle_radian), tf.cos(angle_radian))
],
axis=0
)

rotation_matrix = tf.transpose(rotation_matrix)
landmarks_rotated = tf.linalg.matmul(landmarks, rotation_matrix)

return landmarks_rotated

@tf.function
def tf_atan2(y, x):
angle = tf.where(tf.less(x, 0.0), tf.atan(y / x) + np.pi, tf.atan(y / (x + 1e-12)))
angle = tf.where(tf.less(y, 0.0), angle - (2.0 * np.pi), angle)
angle = tf.where(tf.logical_and(tf.equal(x, 0.0), tf.equal(y, 0.0)), tf.zeros_like(x), angle)
return angle

# Keras model

# Model inputs
input_scalar = tf.keras.layers.Input(shape=(1), dtype=tf.float32, name='handedness')
input_matrix = tf.keras.layers.Input(shape=(21, 2), dtype=tf.float32, name='landmarks')

# Keep first idx of each sample
first_element = tf.gather(input_matrix, 0, axis=1, name='first_element')

# Substract by first element
x = tf.keras.layers.subtract([input_matrix, first_element], name='substract', input_shape=(21, 2))

# Rotation
x = tf.keras.layers.Lambda(tf_rotate_vertically, name='rotate_vertically', input_shape=(21, 2))(x)

preprocessing_model = tf.keras.Model(inputs=input_matrix, outputs=x)

preprocessing_model.predict(tf.reshape(np.random.rand(10, 21, 2), [10, 21, 2])) # OK

# Save model

# Convert the model to Core ML
input_matrix_shape = ct.Shape(shape=(1, 21, 2))

coreml_model = ct.convert(
preprocessing_model,
inputs=[
ct.TensorType(name='landmarks', shape=input_matrix_shape),
],
source='tensorflow',
convert_to="mlprogram"
)

coreml_preprocess_path = '../model/IOS/preprocess_model.mlpackage'

# Save the model
coreml_model.save(coreml_preprocess_path)
```

**KO**
```python
# Random Input to test loaded ml
input = {
"landmarks": np.random.rand(1, 21, 2),
}

# Load the model
model = ct.models.MLModel(coreml_preprocess_path, compute_units=ct.ComputeUnit.CPU_ONLY)
predictions = model.predict(input)
```

## System environment (please complete the following information):
- coremltools version: 7.0
- numpy version: 1.24.3
- tensorflow version: 2.13.0
- OS (e.g. MacOS version or Linux type): M1 Pro 13.6

Contributor guide

Open the contributing guide

Research direction

The issue names no repository files or tests; start by running the provided TensorFlow conversion and MLModel.predict reproduction with coremltools 7.0, TensorFlow 2.13.0, and NumPy 1.24.3. Trace where the loaded mlpackage raises “Error in declaring network”; done means identifying the reproducible cause and whether prediction can work for this model.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.