apple / apple/coremltools

PyTorch flexible input working but not flexible output shapes with 5.0b1

Open
#1,244 12 comments 0 reactions 0 assignees View on GitHub
bug Flexible Shape
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

Even though the produced MLModel shows support for flexible input shapes and output shapes, the output is fixed with PyTorch model converted using the Unified Convertor.

It appears that flexible input is working properly but not the flexible output shape.

If I set the image dimension to 256 and give it an input image of 512 it is cropped to 256.

If I set the image dimension to 2048 and give it an input image of 256, the output is 2048 with mostly blank pixels (just the first 256 pixels are imaged).

An Apple engineer I spoke with at WWDC informed me that flexible images should be properly working with Monterey.

Reproducible with steps: https://github.com/apple/coremltools/issues/992

## System environment
- coremltools 5.0b1
- Mac OS 12.0 Beta 21A5268h
- Versions 12.5 (12E262), 13.0 beta (13A5155e)
- Anaconda
- Python 3.8
- PyTorch 1.9.0

This is a critical issue for my work.

Fixed size is 2048:
![Screen Shot 2021-07-07 at 3 03 03 PM](https://user-images.githubusercontent.com/352383/124828557-aab1bb00-df34-11eb-9efa-9d5e3371c674.png)

Input 256 pixel image:
![Screen Shot 2021-07-07 at 3 03 32 PM](https://user-images.githubusercontent.com/352383/124828589-b3a28c80-df34-11eb-9b00-f74b2db6a54d.png)

Input 1024 pixel image:
![Screen Shot 2021-07-07 at 3 03 50 PM](https://user-images.githubusercontent.com/352383/124828632-c6b55c80-df34-11eb-8126-8ad4e18292a6.png)

Contributor guide

Open the contributing guide

Research direction

Start with the reproducible steps in issue #992 and inspect the PyTorch Unified Converter path for flexible image shapes. Re-run the reported 256, 1024, and 2048 pixel inputs against the generated MLModel. Done means output dimensions follow the input image rather than remaining fixed or being padded with blank pixels.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.