Core ML Model performance far lower on iOS 17 vs iOS 16 (iOS 17 not using Neural Engine)
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## 🐞Describing the bug
Hello,
I'm not sure if this is 100% `coremltools` related or an iOS level issue.
But I've found since iOS 17 release that none of my Core ML models running on devices with iOS 17 use the neural engine, thus resulting in far slower performance.
The following screenshots show the performance of the same model (a PyTorch computer vision model) on an iPhone SE 3rd gen and iPhone 13 Pro (both use the A15 Bionic).
### iOS 16 - iPhone SE 3rd Gen
iOS 16 uses the ANE and results in fast prediction, load and compilation times.
### iOS 17 - iPhone 13 Pro
iOS 17 doesn't seem to use the ANE, thus the prediction, load and compilation times are all slower.
## To Reproduce
The following is my code I'm using to export my PyTorch vision model.
I've used the same code for the past few months with sensational results on iOS 16.
```python
# Convert to Core ML using the Unified Conversion API
coreml_model = ct.convert(
model=traced_model,
inputs=[image_input],
outputs=[ct.TensorType(name="output")],
classifier_config=ct.ClassifierConfig(class_names),
convert_to="neuralnetwork",
# compute_precision=ct.precision.FLOAT16,
compute_units=ct.ComputeUnit.ALL
)
```
## System environment:
- Xcode version: 15.0
- coremltools version: 7.0.0
- OS (e.g. MacOS version or Linux type): Linux Ubuntu 20.04 (for exporting), macOS 13.6 (for testing on Xcode)
- Any other relevant version information (e.g. PyTorch or TensorFlow version): PyTorch 2.0
## Additional context
- This happens across `"neuralnetwork"` and `"mlprogram"` type models, neither use the ANE on iOS 17 but both use the ANE on iOS 16
If anyone has a similar experience, I'd love to hear more.
Otherwise, if I'm doing something wrong for the exporting of models for iOS 17+, please let me know.
Thank you!
Contributor guide
Assessment
This issue has not been assessed yet.