ConvTranspose2d - RuntimeError: PyTorch convert function for op '__is__' not implemented.
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describe the bug
PyTorch model conversion for any model using the layer `ConvTranspose2d` always fails with a runtime error.
`RuntimeError: PyTorch convert function for op '__is__' not implemented.`
## Trace
```
Traceback (most recent call last):
mlmodel = coremltools.converters.convert(
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/_converters_entry.py", line 306, in convert
mlmodel = mil_convert(
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 175, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 202, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 293, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 103, in __call__
return load(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 80, in load
raise e
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 72, in load
prog = converter.convert()
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 231, in convert
convert_nodes(self.context, self.graph)
File "/usr/local/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 66, in convert_nodes
raise RuntimeError(
RuntimeError: PyTorch convert function for op '__is__' not implemented.
```
I will also provide the output of `self.context` and `self.graph` from the `converter` for the example model I provide in the next section:
```
%y.1 : (1, 48, 480, 272, fp32)
%1 : (3,fp32)*
%2 : (48, 3, 3, 3, fp32)*
graph(
%y.1 : Tensor(1, 48, 480, 272, ".double'>"),
%1 : Tensor(3, 'float32'),
%2 : Tensor(48, 3, 3, 3, 'float32'),
):
%3 = constant[]()
%4 = constant[value=requested an output size of {}, but valid sizes range from {} to {} (for an input of {})]()
%5 = constant[value=True]()
%6 = constant[value=output_size must have {} or {} elements (got {})]()
%7 = constant[]()
%8 = constant[value=2]()
%9 = constant[value=0]()
%10 = constant[value=3]()
%11 = constant[value=1]()
%12 = listconstruct[](%8, %8)
%13 = listconstruct[](%9, %9)
%14 = listconstruct[](%10, %10)
%15 = listconstruct[](%11, %11)
%16 = __is__[value=False](%3, %7)
%output_padding.1 = if[](%16)
block():
%ret.1 = listconstruct[](%9, %9)
return (%ret.1)
block():
%output_size.7 = unchecked_cast[](%3)
%20 = dim[](%y.1)
%k.1 = sub[](%20, %8)
%22 = len[](%output_size.7)
%23 = eq[value=False](%22, %20)
%output_size.47 = if[](%23)
block():
%output_size.15 = slice[](%output_size.7, %8, %7, %11)
return (%output_size.15)
block():
return (%output_size.7)
%26 = len[](%output_size.47)
%27 = ne[value=False](%26, %k.1)
= if[](%27)
block():
%28 = len[](%output_size.47)
%29 = format[](%6, %k.1, %20, %28)
= raiseexception[](%29)
return ()
block():
return ()
%min_sizes.1 = listconstruct[]()
%max_sizes.1 = listconstruct[]()
%dilation.17, %53.out, %58.out = loop[](%k.1, %5, %15, %min_sizes.1, %max_sizes.1)
block(%d.1, %dilation.15, %dilation.17.0, %dilation.17.0):
%35 = add[](%d.1, %8)
%36 = size[](%y.1, %35)
%37 = sub[](%36, %11)
%38 = __getitem__[](%12, %d.1)
%39 = mul[](%37, %38)
%40 = __getitem__[](%13, %d.1)
%41 = mul[](%40, %8)
%42 = sub[](%39, %41)
%43 = __isnot__[value=False](%dilation.15, %7)
%44, %dilation.13 = if[](%43)
block():
%dilation.7 = unchecked_cast[](%dilation.15)
%47 = __getitem__[](%dilation.7, %d.1)
return (%47, %dilation.7)
block():
return (%11, %dilation.15)
%48 = __getitem__[](%14, %d.1)
%49 = sub[](%48, %11)
%50 = mul[](%44, %49)
%51 = add[](%42, %50)
%dim_size.1 = add[](%51, %11)
%53 = append[](%dilation.17.0, %dim_size.1)
%54 = __getitem__[](%53.out, %d.1)
%55 = __getitem__[](%12, %d.1)
%56 = add[](%54, %55)
%57 = sub[](%56, %11)
%58 = append[](%dilation.17.0, %57)
return (%5, %dilation.13, %53, %58)
%59 = len[](%output_size.47)
= loop[](%59, %5)
block(%i.1):
%size.1 = __getitem__[](%output_size.47, %i.1)
%min_size.1 = __getitem__[](%53.out, %i.1)
%max_size.1 = __getitem__[](%58.out, %i.1)
%64 = lt[value=False](%size.1, %min_size.1)
%65 = if[value=False](%64)
block():
return (%5)
block():
%66 = gt[value=False](%size.1, %max_size.1)
return (%66)
= if[](%65)
block():
%67 = size[](%y.1)
%68 = slice[](%67, %8, %7, %11)
%69 = format[](%4, %output_size.47, %53.out, %58.out, %68)
= raiseexception[](%69)
return ()
block():
return ()
return (%5)
%res.1 = listconstruct[]()
%75.out = loop[](%k.1, %5, %res.1)
block(%d.17, %loop.1.0):
%72 = __getitem__[](%output_size.47, %d.17)
%73 = __getitem__[](%53.out, %d.17)
%74 = sub[](%72, %73)
%75 = append[](%loop.1.0, %74)
return (%5, %75)
return (%75.out)
%76 = listconstruct[](%8, %8)
%77 = listconstruct[](%9, %9)
%78 = listconstruct[](%11, %11)
%z.1 = conv_transpose2d[](%y.1, %2, %1, %76, %77, %output_padding.1, %11, %78)
```
Note ` %16 = __is__[value=False](%3, %7)` .
## To Reproduce
Define a PyTorch model with `ConvTranspose2d`:
```
class MyModel(nn.Module):
def __init__(self):
super().__init__()
self.conv = nn.ConvTranspose2d(48, 3, 3, 2)
def forward(self, y):
output = self.conv(y)
return output
````
Convert the model in the standard way:
```
input_shape = torch.Size([1,48,480,272])
torchscript_model = torch.jit.script(model)
mlmodel = coremltools.converters.convert(
torchscript_model,
inputs=coremltools.TensorType(name='input', shape=input_shape),
# triggers the runtime error
)
```
## System environment:
- coremltools version: 5.0 (from pip)
- OS: Linux
- How you install python: from source
- python version: 3.8.5
- pytorch version: 1.9.0 (from pip)
Contributor guide
Research direction
Start with the PyTorch conversion path in coremltools/converters/mil/frontend/torch/load.py and converter.py, then inspect convert_nodes in coremltools/converters/mil/frontend/torch/ops.py. Reproduce the failure with the provided scripted ConvTranspose2d model and trace the generated __is__ operation. Done means the model converts successfully without the reported RuntimeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100