ml-explore / ml-explore/mlx-examples
Enable the Mixtral-like Moe model without the quantized gate layer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Currently, the community has started experimenting with building more models using a mix of different local experts. In the current implementation of mlx-lm, we have hardcoded the linear_class_predicate with 8 as an assumption that there will always be 8 local experts. It would be great if we could read the number of local experts from the configuration and make it adjustable to support varying numbers of local experts. This way, users can use mlx-lm to fine-tune the gate for LoRa. For example, here I have to copy and paste code from mlx-lm in order to make lora moe 4x7b work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the mlx-lm implementation of linear_class_predicate and the model configuration it currently uses, then compare it with the duplicated logic in utils.py lines 57-66 of the linked mlx-moe example. Done means the number of local experts is read from configuration rather than fixed at 8, while supporting LoRA fine-tuning for varying expert counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100