apple / apple/coremltools

Output specifications support for Pytorch converter

Open
#775 12 comments 13 reactions 0 assignees View on GitHub
feature request PyTorch (traced)
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🌱 Describe your Feature Request

As the documentation explicitly point it out: "it is good practice to think about the interface to the model. This includes names and types of inputs and outputs".
When converting a model from PyTorch, one can just not explicitly set output type, name and size. Those specifications are also very useful when we need to scale / preprocess inputs and outputs.

## Use case
In my case I need my model to output an imageType instead of a multiArray. I'd like to provide specifications like this:
```
mlmodel = coremltools.convert(model,
inputs=[ct.ImageType(name="input", shape=(1, 3, 1024, 1024), scale=1/255)],
outputs=[ct.ImageType(name="output", shape=(1, 3, 1024, 1024), scale=255)])
```
but get the following error : `ValueError: outputs must not be specified for PyTorch`

## Alternative
The alternative to get the output specifications right is to manually change them through the mlmodel specs :

1) Specify multiArray shapes
```
spec = mlmodel.get_spec()
spec.description.output[0].type.multiArrayType.shape.append(3)
spec.description.output[0].type.multiArrayType.shape.append(1024)
spec.description.output[0].type.multiArrayType.shape.append(1024)
```

2) Set output type to imageType

```
from coremltools.proto import FeatureTypes_pb2 as ft

for output in spec.description.output:

# Change output type
output.type.imageType.colorSpace = ft.ImageFeatureType.ColorSpace.Value('RGB')
channels, height, width = tuple(output.type.multiArrayType.shape)

# Set image shape
output.type.imageType.width = width
output.type.imageType.height = height
```

3) Convert back the modified specs into a new mlmodel

`mlmodel_modif = coremltools.models.MLModel(spec)`

But when calling `predict` on this new model i got the following error:
```
RuntimeError: {
NSLocalizedDescription = "Batch or sequence image output is unsupported for image output 1223";
}
```

## System environment :
- coremltools version 4.0b1
- Pytorch 1.5
- MacOs 10.15.4
- XCode 11.5
- Python 3.7

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.