AOT perf regression on Apple platform
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🐛 Describe the bug
Over the weekend I start observing timeout when exporting models.
Affected models:
- mv3
- mv2
Affected configs: - xnnpack_q8
- coreml_fp16
Affected platforms:
- MocOS only.
Expected:
The export jobs used to finish within 30mins
Actual:
Some jobs start hitting the timeout threshold 60min since 1/25
1/26
- mv3, xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12980356644/job/36197276541
- mv3, coreml_fp16: 60min: https://github.com/pytorch/executorch/actions/runs/12980356644/job/36197276201
1/25
- mv3 xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12970194466/job/36175156930
- mv2 xnnpack_q8: 60min: https://github.com/pytorch/executorch/actions/runs/12970194466/job/36175156520
View historical runs of (mv3, xnnpack_q8) on macOS: https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=apple-perf%20%2F%20export-models%20(mv3%2C%20xnnpack&mergeLF=true
View historical runs of (mv3, coreml_fp16) on macOS: https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=apple-perf%20%2F%20export-models%20(mv3%2C%20coreml&mergeLF=true
Use this query to view all details from the scheduled runs: https://github.com/pytorch/executorch/actions/workflows/apple-perf.yml?query=event%3Aschedule
Versions
trunk
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the scheduled runs from apple-perf.yml and compare the affected macOS jobs against their historical HUD results. Investigate the regression affecting mv2/mv3 exports with xnnpack_q8 and coreml_fp16; done means the affected export jobs complete within the expected 30-minute window.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100