Issues during CoreML GA model testing
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🐛 Describe the bug
Repro steps:
Follow the repro steps here for exporting/running GA models.
Export issues:
- When take_over_mutable_buffer=True, smollm fails with "Buffer output getitem_1 does not point to a buffer that exist", see smollm_coreml_executorch_exception.txt.
- Model does not quantize using default iOS15, see yolos-quantize_coreml_executorch_exception.txt
Runtime issues, see run_logs.txt
- "[ETCoreMLModelManager.mm:566] [Core ML] Metadata is invalid or missing.
[backend_delegate.mm:288] [Core ML] Model init failed Metadata is invalid or missing."
We see this issue in various models like whisper2, sam_vision_encoder, yolos2.
Performance issues:
- Quantized version of yolos yolos-quantize-ios17 is almost 2x slower than non-quantized version.
Versions
NA
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the repro steps in ga_testing.md and review run_logs.txt alongside the linked export exception files. Separate the mutable-buffer and quantization export failures from the Core ML metadata runtime failures and the YOLOS performance result. Done means the listed model tests no longer show those failures and the quantized YOLOS runtime result is investigated or corrected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ios
- Domain
- machine-learning, mobile-dev, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100