Mac running very slow
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
I have the Macbook Pro 2023 M2 Max version with 38 GPUs and 96GB MEM, but running even 7B is very slow. The GPUs is not fully utilized, less than 20% most of the time. Generating 1 token takes 2~3 seconds. Anything I can do to improve the speed?
BTW, the Apple Neural Engine is not working at all
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported 2–3 seconds per token behavior on the specified MacBook Pro M2 Max with a 7B model, while checking GPU and Apple Neural Engine utilization. The issue is resolved when generation speed improves or the cause of the low utilization is identified and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100