ml-explore / ml-explore/mlx-examples
Performance with M1 Pro 16GB: Is it Normal?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hello, can you provide a minimum configuration for model usage?
macOS 13.4.1 14-inch M1 Pro 16GB
- LLaMA. It takes so long time to chat and not available in fact.
(mlx) llama % python3 llama.py Llama-2-7b-chat.npz tokenizer.model "who are you?"
[INFO] Loading model from disk.
Press enter to start generation
------
The memory consumption reaches around 13GB.
2. Stable Diffusion
(mlx) stable_diffusion % python3 txt2image.py "a beautiful flower" --output flower.png
2%|█▊ | 1/50 [00:20<16:55, 20.72s/it]
The memory consumption reaches around 11GB , and it takes more than ten mins.
Unfortunately, given these observations, it seems that the mlx framework is almost unavailable for machines with 16GB M1 Pro.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported commands in llama/llama.py and stable_diffusion/txt2image.py on the stated macOS and M1 Pro configuration. Compare the observed memory use and generation times with the examples' documented expectations, then establish whether a minimum configuration or performance guidance can be stated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100