ml-explore / ml-explore/mlx-examples
GaLore process on Apple Silicon?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
I have just read the very recent paper [GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection](arXiv 2403:03507) that allows Llama 7B training to be run on an RTX 4090 in 24GB of VRAM. On the face of it this suggests that the same technique could be applied on Apple Silicon, but I don't know enough about the relative parameters of the 4090 and M1, M2, M3 chips to know how tractable that is, or whether the MLX project yet supports sufficient Torch-like ops.
Can anyone comment? Should a 32GB (38 core) M2 MAX be able to do the same thing? How long would it take?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the GaLore paper's required operations with the MLX project's available Torch-like operations on Apple Silicon. Determine whether a 32GB M2 Max can support the described Llama 7B training and document the expected feasibility and training time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100