Output specifications support for Pytorch converter
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🌱 Describe your Feature Request
As the documentation explicitly point it out: "it is good practice to think about the interface to the model. This includes names and types of inputs and outputs".
When converting a model from PyTorch, one can just not explicitly set output type, name and size. Those specifications are also very useful when we need to scale / preprocess inputs and outputs.
## Use case
In my case I need my model to output an imageType instead of a multiArray. I'd like to provide specifications like this:
```
mlmodel = coremltools.convert(model,
inputs=[ct.ImageType(name="input", shape=(1, 3, 1024, 1024), scale=1/255)],
outputs=[ct.ImageType(name="output", shape=(1, 3, 1024, 1024), scale=255)])
```
but get the following error : `ValueError: outputs must not be specified for PyTorch`
## Alternative
The alternative to get the output specifications right is to manually change them through the mlmodel specs :
1) Specify multiArray shapes
```
spec = mlmodel.get_spec()
spec.description.output[0].type.multiArrayType.shape.append(3)
spec.description.output[0].type.multiArrayType.shape.append(1024)
spec.description.output[0].type.multiArrayType.shape.append(1024)
```
2) Set output type to imageType
```
from coremltools.proto import FeatureTypes_pb2 as ft
for output in spec.description.output:
# Change output type
output.type.imageType.colorSpace = ft.ImageFeatureType.ColorSpace.Value('RGB')
channels, height, width = tuple(output.type.multiArrayType.shape)
# Set image shape
output.type.imageType.width = width
output.type.imageType.height = height
```
3) Convert back the modified specs into a new mlmodel
`mlmodel_modif = coremltools.models.MLModel(spec)`
But when calling `predict` on this new model i got the following error:
```
RuntimeError: {
NSLocalizedDescription = "Batch or sequence image output is unsupported for image output 1223";
}
```
## System environment :
- coremltools version 4.0b1
- Pytorch 1.5
- MacOs 10.15.4
- XCode 11.5
- Python 3.7
Contributor guide
Assessment
This issue has not been assessed yet.