apple / apple/coremltools

Transpose Op seems useless and lead to low performance

Open
#985 5 comments 7 reactions 0 assignees View on GitHub
feature request question triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question

I have designed a simple network using tensorflow 1.15, and converted it to mlmodels using [tf-coreml ](https://github.com/tf-coreml/tf-coreml)and coremltools4.0. I have tested them on iPhoneXS with system version ios13.5.1, and found that the mlmodel converted from coremltools4.0 is much slower than from tfcoreml, about 2x slower.

The original network in tensorflow is as following:
```
graph = tf.Graph()
with graph.as_default():
x = tf.placeholder(tf.float32, shape=[1, 1000, 1000, 4], name="input")
y = tf.layers.conv2d(x, 4, 1, padding='same', activation=tf.nn.relu)
output_names = [y.op.name]
```

1. Converting to mlmodel using tfcoreml
```
# using tfcoreml
coreml_save_tfcoreml_file = model_dir + "/debug_tfcoreml.mlmodel"
tfcoreml.convert(tf_model_path=frozen_graph_file,
mlmodel_path=coreml_save_tfcoreml_file,
output_feature_names=["conv2d/Relu:0"], # name of the output tensor (appended by ":0")
input_name_shape_dict={"input": [1, 1000, 1000, 4]}, # input tensor[1, height, width, channel]
minimum_ios_deployment_target='12')
```
![image](https://user-images.githubusercontent.com/26928415/98205661-7a63e800-1f73-11eb-8a1d-2b6c74475c13.png)

2. Converting to mlmodel using coremltools4.0
```
coreml_save_coremltools_file = model_dir + "/debug_coremltools.mlmodel"
mlmodel = ct.convert(frozen_graph_file, source='tensorflow')
mlmodel.save(coreml_save_coremltools_file)
```
![image](https://user-images.githubusercontent.com/26928415/98205640-71731680-1f73-11eb-8e9c-c766e9785f36.png)

I have tried to use coremltool to remove the two transpose layers and change input/output nodes' shape info from HWC to CHW. Then the model's performance from coremltools and tfcoreml become same.

It seems that the two transpose layers are really time-cost and they can not run on ANE or gpu. So why coremltools does not adopt the NCHW format directly?

Futhermore, I also found that the **resize_bilinear** op in tensorflow is mapped to **upsample** op in tf-coreml, but mapped to **resizeBilinear** op in coremltools4.0. However, resizeBilinear op can not run on apple's ANE, but upsample op can run on ANE. Why coremltools does not use the upsample op directly?

Thanks.

## System Information
- If applicable

macos catalina 10.15
iPhoneXS ios13.5.1
coremltools 4.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.