apple / apple/coremltools

4.0b2: Flexible shapes not working but enumerated image sizes work

Open
#840 4 comments 0 reactions 0 assignees View on GitHub
bug Flexible Shape
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Bug
I have successfully converted a model to use images instead of multi-arrays (with 4.0b2).

I can add flexible shapes and the model compiles/exports fine. If I run predict on the resulting mode with an input image with a dimension that is not the fixed dimension (and inside the flexible shape range), I get the trace below:

## Trace
> Traceback (most recent call last):
> File "/Users/jeshua/produceImageWithMLModel.py", line 36, in
> main()
> File "/Users/jeshua/produceImageWithMLModel.py", line 30, in main
> outputImage = model.predict({args.inputLayer: image})[args.outputLayer]
> File "/Users/jeshua/coremltools4/lib/python3.8/site-packages/coremltools/models/model.py", line 329, in predict
> return self.__proxy__.predict(data, useCPUOnly)
> RuntimeError: {
> NSLocalizedDescription = "Error binding image input buffer input.";

## Code snippet
I added the flexible shapes like this:
```
img_size_ranges = flexible_shape_utils.NeuralNetworkImageSizeRange()
img_size_ranges.add_height_range((64, 4096))
img_size_ranges.add_width_range((64, 4096))

flexible_shape_utils.update_image_size_range(spec, feature_name='input', size_range=img_size_ranges)
flexible_shape_utils.update_image_size_range(spec, feature_name='output', size_range=img_size_ranges)
```

If I add enumerated shapes, the enumerated shapes work (only):

```
image_sizes = [flexible_shape_utils.NeuralNetworkImageSize(512, 512)]
image_sizes.append(flexible_shape_utils.NeuralNetworkImageSize(1024, 1024))
image_sizes.append(flexible_shape_utils.NeuralNetworkImageSize(2048, 2048))
flexible_shape_utils.add_enumerated_image_sizes(spec, feature_name='input', sizes=image_sizes)
flexible_shape_utils.add_enumerated_image_sizes(spec, feature_name='output', sizes=image_sizes)
```

## System environment:
- coremltools version 4.0b2
- OS both MacOS, Linux
- macOS version 10.15.5
- virtualenv
- python version 3.6

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure using the flexible_shape_utils calls in the issue and an in-range image size that is not fixed. Start by tracing image input binding during model.predict; done means flexible image ranges work on both input and output without the reported binding error, while enumerated sizes remain supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.