apple / apple/coremltools

Can't Convert Keras Model with Flexible Input Length and Dilated Convolution(s)

Open
#1,389 5 comments 0 reactions 0 assignees View on GitHub
bug tf2.x / tf.keras triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question
I have a keras model that contains some dilated convolutions and whose input shape contains a variable component. When trying to convert the model using `coremltools.convert()`, I get the following error: `Not support dynamic block_shape and paddings for SpaceToBatchND!`. If I specify the variable component in the input shape, I get this error instead: `non-equal block shape is not yet supported for 4d input.`. I managed to create two very simple examples that showcase these two problems:

1. Variable Input Length
```python
import tensorflow as tf
import coremltools as ct

inputs = tf.keras.layers.Input(shape=(1, None, 1))
dilated_conv = tf.keras.layers.Conv1D(filters=20, kernel_size=5, dilation_rate=2)(inputs)
model = tf.keras.Model(inputs=inputs, outputs=[dilated_conv])

ct.convert(model) # Fails with error: "Not support dynamic block_shape and paddings for SpaceToBatchND!"
```

2. Fixed Input Length
```python
import tensorflow as tf
import coremltools as ct

inputs = tf.keras.layers.Input(shape=(1, 300, 1))
dilated_conv = tf.keras.layers.Conv1D(filters=20, kernel_size=5, dilation_rate=2)(inputs)
model = tf.keras.Model(inputs=inputs, outputs=[dilated_conv])

ct.convert(model) # Fails with error: "non-equal block shape is not yet supported for 4d input."
```

I've done some extensive debugging but I have no idea how to approach this. Initially, padding was set to 'same', which I thought may have been causing the issue; so I tried adding a padding layer before the convolution instead of specifying it when creating the layer. But as you can see from the examples above, it fails even if padding isn't specified at all.
I would appreciate any help on this issue! :)

## System Information
`coremltools.__version__ == '5.1.0'`
`tensorflow.__version__=='2.5.0'`

Contributor guide

Open the contributing guide

Research direction

Start by running the two minimal Keras and coremltools conversion examples against the reported TensorFlow 2.5.0 and coremltools 5.1.0 versions, then trace the reported SpaceToBatchND errors in the converter. Done means both variable-length and fixed-length dilated-convolution models convert successfully, with regression coverage for the two cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.