apple / apple/coremltools

coremltools does not export LSTM state?

Open
#1,966 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question

I am trying to convert an LSTM based model from tensorflow to CoreML to be used in a MacOS application. Despite of looking at hundreds of examples in documentation, and spending two days through all possible references to a similar problem, I can't find a solution. Here is a compact example that replicates the problem.

First we define a very simple LSTM model and export it to CoreML using coremltools:
```
import os
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow import keras
import coremltools

model = keras.Sequential()
model.add(layers.Input(dtype="float32", batch_input_shape=(1,1,1)))
model.add(layers.LSTM(3,))
model.add(layers.Dense(1))
model.summary()

coreml_model_file = './test.mlpackage'

mlmodel = coremltools.convert(model,source="tensorflow")
mlmodel.save(coreml_model_file)
```
tensorflow version is 2.13.0, coremltools version is 6.3.0, python version 3.11. Tried with tensorflow 2.12.0, does not change a thing.

Dragging .mlpackage to Xcode performs perfect import. But I get the following as model specification: [SCREENSHOT OF MY MODEL](https://i.stack.imgur.com/HXQ0P.png)

Documentation (https://developer.apple.com/documentation/coreml/making_predictions_with_a_sequence_of_inputs?language=objc) suggests that the states of the LSTM layer should be exported as additional inputs and outputs, so the network can be applied to a new arbitrary sequence. Whatever I do I cannot get statein/stateout to appear in CoreML model.

What am I doing wrong? How do I export an LSTM model properly?

Tried different versions of tensorflow 2, coremltools. Stateful LSTM does not export at all. Changing input shape to (1, None, 1) does not help either.

Will really appreciate some help.

Contributor guide

Open the contributing guide

Research direction

Start by running the compact TensorFlow-to-Core ML conversion example with the reported TensorFlow 2.13.0, coremltools 6.3.0, and Python 3.11 versions, then inspect the generated model specification in Xcode. Compare the result with Apple's sequence-input documentation and determine whether statein/stateout should be present. Done means identifying a supported export path or documenting the limitation and required conditions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.