apple / apple/coremltools

ValueError: "ResizeBilinear" op: the second input, which is the output size, must be known statically

Open
#743 2 comments 3 reactions 0 assignees View on GitHub
bug tf1.x
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describe the bug
I am using `CoreMLTools 4.0b1` and `Tensorflow 1.14`.
I am trying to convert the [mask_rcnn_resnet50_atrous_coco](http://download.tensorflow.org/models/object_detection/mask_rcnn_resnet50_atrous_coco_2018_01_28.tar.gz) model from Tensorflow model zoo.

I am getting the error:
`ValueError: "ResizeBilinear" op: the second input, which is the output size, must be known statically`

## Trace
```
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Running TensorFlow Graph Passes: 33%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 2/6 [00:01<00:02, 1.71 passes/s]WARNING:root:[Constant Propagation] Skip "dead" tensor: BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_f:0
WARNING:root:[Constant Propagation] Skip "dead" tensor: BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_t:0
WARNING:root:[Constant Propagation] Skip "dead" tensor: BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/Switch:0
ERROR:root:[None, True]
ERROR:root:BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_t:0
ERROR:root:None
ERROR:root:BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_f:0
ERROR:root:None
WARNING:root:[Constant Propagation] Skip "dead" tensor: SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_f:0
WARNING:root:[Constant Propagation] Skip "dead" tensor: SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/Switch:0
WARNING:root:[Constant Propagation] Skip "dead" tensor: SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_t:0
WARNING:root:[Constant Propagation] Skip "dead" tensor: SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond_3/cond/Switch:1
WARNING:root:[Constant Propagation] Skip "dead" tensor: SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond_1/cond/Switch:1
ERROR:root:[None, True]
ERROR:root:SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_t:0
ERROR:root:None
ERROR:root:SecondStagePostprocessor/BatchMultiClassNonMaxSuppression/map/while/PadOrClipBoxList/cond/cond/switch_f:0
ERROR:root:None
ERROR:root:[False, None]
ERROR:root:[False, None]
Running TensorFlow Graph Passes: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 6/6 [00:26<00:00, 4.48s/ passes]
Converting Frontend ==> MIL Ops: 39%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 437/1117 [00:00<00:01, 527.88 ops/sWARNING:root:Input ls elem type unknown. Override with .tensor'> | 0/75 [00:00 MIL Ops: 93%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 70/75 [00:00<00:00, 909.54 ops/s]
Converting Frontend ==> MIL Ops: 41%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 463/1117 [00:00<00:01, 526.69 ops/s]
Traceback (most recent call last):
File "convert.py", line 92, in
mlmodel = ct.convert(model=path, inputs=[model_input])
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/_converters_entry.py", line 258, in convert
**kwargs
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 120, in _convert
prog = frontend_converter(model, **kwargs)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/converter.py", line 41, in __call__
return tf1_loader.load()
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 79, in load
program = self._program_from_tf_ssa()
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/load.py", line 217, in _program_from_tf_ssa
return converter.convert()
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 389, in convert
self.convert_main_graph(prog, graph)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 334, in convert_main_graph
outputs = convert_graph(self.context, graph, self.outputs)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 181, in convert_graph
_add_op(context, node)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 2173, in While
x = mb.while_loop(_cond=cond, _body=body, loop_vars=loop_vars, name=node.name)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
return cls._add_op(op_cls, **kwargs)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/mil/builder.py", line 190, in _add_op
new_op.build_nested_blocks()
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/mil/ops/defs/control_flow.py", line 247, in build_nested_blocks
exit_vars = body_func(*block.inputs)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 2168, in body
res = convert_graph(context, body_graph)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/convert_utils.py", line 181, in convert_graph
_add_op(context, node)
File "/Users/fotidim/.local/share/virtualenvs/models/lib/python3.7/site-packages/coremltools/converters/mil/frontend/tensorflow/ops.py", line 2048, in ResizeBilinear
'"ResizeBilinear" op: the second input, which is the output size, must be known statically'
ValueError: "ResizeBilinear" op: the second input, which is the output size, must be known statically
```

## To Reproduce

```
import coremltools as ct
import tensorflow as tf

path = 'frozen_inference_graph.pb'

model_input = ct.ImageType(shape=(1, 600, 600, 3)) # or ct.ImageType(shape=(1, -1, -1, 3))
mlmodel = ct.convert(model=path, inputs=[model_input])
mlmodel.save(graph.replace("pb", "mlmodel"))
```

- Source model
`http://download.tensorflow.org/models/object_detection/mask_rcnn_resnet50_atrous_coco_2018_01_28.tar.gz`

## System environment (please complete the following information):
- coremltools version: 4.0b1
- OS: MacOS,
- macOS version: 10.15.5 (19F101)
- How you install python: brew and pipenv
- python version: 3.7.6
- Tensorflow version: 1.1.4

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the supplied Mask R-CNN model and conversion command, then inspect coremltools/converters/mil/frontend/tensorflow/ops.py at ResizeBilinear and the control-flow path in converter.py. Done means the model conversion completes without the reported static output-size ValueError and the behavior is covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
devtools, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.