apple / apple/coremltools

In TF_KERAS conversion ConvTransposed with a weight matrix ndims!=4 not working

Open
#1,038 2 comments 0 reactions 0 assignees View on GitHub
bug tf2.x / tf.keras triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describe the bug
Converting a tensorflow keras model with Conv2DTranspose layer with a weight matrix having 3 dimensions fails.

## Trace
2020-12-22 09:53:45.124910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-22 09:53:45.124915: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0
2020-12-22 09:53:45.124920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N
Running TensorFlow Graph Passes: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 5/5 [00:00<00:00, 95.59 passes/s]
Converting Frontend ==> MIL Ops: 85%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 11/13 [00:00<00:00, 4040.40 ops/s]
Traceback (most recent call last):
File "tf_to_coreml_conversion_bug.py", line 22, in
coreml_model = ct.convert(keras_model, inputs=[model_input])
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/_converters_entry.py", line 265, in convert
**kwargs
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/converter.py", line 134, in _convert
prog = frontend_converter(model, **kwargs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/converter.py", line 74, in __call__
return tf2_loader.load()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 80, in load
program = self._program_from_tf_ssa()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow2/load.py", line 178, in _program_from_tf_ssa
return converter.convert()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 405, in convert
self.convert_main_graph(prog, graph)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 334, in convert_main_graph
outputs = convert_graph(self.context, graph, self.outputs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 181, in convert_graph
_add_op(context, node)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 1991, in Conv2DBackpropInput
x = _transpose_NHWC_to_NCHW(x)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 45, in _transpose_NHWC_to_NCHW
return mb.transpose(x=x, perm=[0, 3, 1, 2])
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
return cls._add_op(op_cls, **kwargs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/builder.py", line 193, in _add_op
new_op.type_value_inference()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/operation.py", line 189, in type_value_inference
output_types = self.type_inference()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/ops/defs/tensor_transformation.py", line 786, in type_inference
raise ValueError(msg.format(len(perm), self.x.rank))
ValueError: perm should have the same length as rank(x): 4 != 3

## To Reproduce
```python
import tensorflow as tf
import coremltools as ct

def build_model():
model_input = tf.keras.Input(shape=(144, 256, 4))
model_output = tf.keras.layers.Conv2DTranspose(3, (3, 3), strides=(2, 2), padding='same')(model_input)
return tf.keras.models.Model(inputs=model_input, outputs=model_output)

keras_model = build_model()
input_shape = ct.Shape(shape=(144, 256, 4))
model_input = ct.TensorType(shape=input_shape)
coreml_model = ct.convert(keras_model, inputs=[model_input])
```
## System environment (please complete the following information):
- coremltools version: 4.0b4
- tensorflow version: 2.3.0
- OS: Ubuntu 18.04
- python version: 3.6.9

Contributor guide

Open the contributing guide

Research direction

Start by running the provided TensorFlow and coremltools reproduction with the stated model shape and inspect the Conv2DBackpropInput path in coremltools/converters/mil/frontend/tensorflow/ops.py. Verify the conversion no longer raises the rank-mismatch ValueError for the Conv2DTranspose layer, then rerun the reproduction to confirm successful conversion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.