In TF_KERAS conversion ConvTransposed with a weight matrix ndims!=4 not working
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## πDescribe the bug
Converting a tensorflow keras model with Conv2DTranspose layer with a weight matrix having 3 dimensions fails.
## Trace
2020-12-22 09:53:45.124910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-12-22 09:53:45.124915: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0
2020-12-22 09:53:45.124920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N
Running TensorFlow Graph Passes: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ| 5/5 [00:00<00:00, 95.59 passes/s]
Converting Frontend ==> MIL Ops: 85%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | 11/13 [00:00<00:00, 4040.40 ops/s]
Traceback (most recent call last):
File "tf_to_coreml_conversion_bug.py", line 22, in
coreml_model = ct.convert(keras_model, inputs=[model_input])
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/_converters_entry.py", line 265, in convert
**kwargs
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/converter.py", line 134, in _convert
prog = frontend_converter(model, **kwargs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/converter.py", line 74, in __call__
return tf2_loader.load()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 80, in load
program = self._program_from_tf_ssa()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow2/load.py", line 178, in _program_from_tf_ssa
return converter.convert()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 405, in convert
self.convert_main_graph(prog, graph)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 334, in convert_main_graph
outputs = convert_graph(self.context, graph, self.outputs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 181, in convert_graph
_add_op(context, node)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 1991, in Conv2DBackpropInput
x = _transpose_NHWC_to_NCHW(x)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 45, in _transpose_NHWC_to_NCHW
return mb.transpose(x=x, perm=[0, 3, 1, 2])
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
return cls._add_op(op_cls, **kwargs)
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/builder.py", line 193, in _add_op
new_op.type_value_inference()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/operation.py", line 189, in type_value_inference
output_types = self.type_inference()
File "/home/alexander/.local/lib/python3.6/site-packages/coremltools/converters/mil/mil/ops/defs/tensor_transformation.py", line 786, in type_inference
raise ValueError(msg.format(len(perm), self.x.rank))
ValueError: perm should have the same length as rank(x): 4 != 3
## To Reproduce
```python
import tensorflow as tf
import coremltools as ct
def build_model():
model_input = tf.keras.Input(shape=(144, 256, 4))
model_output = tf.keras.layers.Conv2DTranspose(3, (3, 3), strides=(2, 2), padding='same')(model_input)
return tf.keras.models.Model(inputs=model_input, outputs=model_output)
keras_model = build_model()
input_shape = ct.Shape(shape=(144, 256, 4))
model_input = ct.TensorType(shape=input_shape)
coreml_model = ct.convert(keras_model, inputs=[model_input])
```
## System environment (please complete the following information):
- coremltools version: 4.0b4
- tensorflow version: 2.3.0
- OS: Ubuntu 18.04
- python version: 3.6.9
Contributor guide
Research direction
Start by running the provided TensorFlow and coremltools reproduction with the stated model shape and inspect the Conv2DBackpropInput path in coremltools/converters/mil/frontend/tensorflow/ops.py. Verify the conversion no longer raises the rank-mismatch ValueError for the Conv2DTranspose layer, then rerun the reproduction to confirm successful conversion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100