apple / apple/coremltools

Converting forward function with multiple outputs causing confused names

Open
#1,887 4 comments 0 reactions 0 assignees View on GitHub
bug PyTorch (traced) triaged
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
When converting PyTorch model to CoreML model, the converting tool seems to be confused with the naming of multiple outputs of the forward function.

## Stack Trace

**Incorrect output**
2 outputs have the same name: input_5

```
/Users/philip/miniconda/envs/pytorch/lib/python3.9/site-packages/coremltools/models/model.py:154: RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was: Error compiling model: "compiler error: Error reading protobuf spec. validator error: Interface specifies output 'input_5' which is not produced by any layer in the neural network.".
_warnings.warn(
input {
name: "x"
type {
multiArrayType {
shape: 1
shape: 3
shape: 224
shape: 224
dataType: FLOAT32
}
}
}
output {
name: "input_5"
type {
multiArrayType {
dataType: FLOAT32
}
}
}
output {
name: "input_5"
type {
multiArrayType {
dataType: FLOAT32
}
}
}
metadata {
userDefined {
key: "com.github.apple.coremltools.source"
value: "torch==2.0.0"
}
userDefined {
key: "com.github.apple.coremltools.version"
value: "7.0b1"
}
}
```

**Correct output**
2 outputs have different names: input_5 and var_24

```
input {
name: "x"
type {
multiArrayType {
shape: 1
shape: 3
shape: 224
shape: 224
dataType: FLOAT32
}
}
}
output {
name: "input_5"
type {
multiArrayType {
dataType: FLOAT32
}
}
}
output {
name: "var_24"
type {
multiArrayType {
dataType: FLOAT32
}
}
}
metadata {
userDefined {
key: "com.github.apple.coremltools.source"
value: "torch==2.0.0"
}
userDefined {
key: "com.github.apple.coremltools.version"
value: "7.0b1"
}
}
```
## To Reproduce
```
import torch
import torch.nn as nn
import torch.nn.functional as F
import coremltools as ct
# Define a simple layer module we'll reuse in our network.

class Layer(nn.Module):
def __init__(self, dims):
super(Layer, self).__init__()
self.conv1 = nn.Conv2d(*dims)

def forward(self, x):
x = F.relu(self.conv1(x))
x = F.max_pool2d(x, (2, 2))
return x
# A simple network consisting of several base layers.
class SimpleNet(nn.Module):
def __init__(self):
super(SimpleNet, self).__init__()
self.layer1 = Layer((3, 6, 3))
self.layer2 = Layer((6, 16, 1))

def forward(self, x):
x = self.layer1(x)
y = self.layer2(x)

# outputs different names for x and y
# y = x + 1

# outputs the same name for x and y, generating an error
y = x

# still produces error even if a new object is created
# y = x.clone().detach()
# print(id(x))
# print(id(y))

return x,y

model = SimpleNet() # Instantiate the network.
example = torch.rand(1, 3, 224, 224) # Example input, needed by jit tracer.
traced = torch.jit.trace(model, example) # Generate TorchScript by tracing.
mlmodel = ct.converters.convert(traced, inputs=[ct.TensorType(shape=example.shape)])
print(mlmodel)

```

## System environment (please complete the following information):
- coremltools version: 7.0b1
- OS (e.g. MacOS version or Linux type): Mac OS 13.3.1 (a)
- Any other relevant version information (e.g. PyTorch or TensorFlow version): torch==2.0.0

Contributor guide

Open the contributing guide

Research direction

Start with the PyTorch conversion path exercised by ct.converters.convert after torch.jit.trace, using the SimpleNet reproducer in the issue. Compare conversion when the two returned tensors share the same object with the distinct-output case, then rerun the reproducer to confirm the generated model gives the outputs unique names and can be validated or used for prediction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.