Question re. ANE Usage with Flexible Input Shapes
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## ❓Question
Not sure if this is a framework issue, or one with `coremltools`. My hunch is the latter, so I'm asking here.
I've exported a model that requires a flexible input shape, and set the default shape to `1`. This model doesn't use the ANE at all, and only runs on CPU.
Out of curiosity, I set the input shape to be fixed to `1` to see if the model would run faster. This model uses the GPU / ANE and is _significantly faster_. Does this mean that ANE usage is out of the window with flexible input shapes, or is there scope to redefine the model to allow it to use the ANE with flexible shapes too?
Unfortunately, I cannot share the model definition publicly.
### Fixed input shape:

### Flexible input shape:

Contributor guide
Research direction
No source file, test, or entry point is named. Start by reproducing the reported performance difference with equivalent fixed and flexible input shapes, then investigate whether coremltools or the exported model controls ANE eligibility; done means determining whether flexible shapes can use the ANE and documenting the supported behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100