facebookresearch / facebookresearch/fairseq2
Support for MoE architecture
Open
enhancement
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 144
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 1
Description
Do you have a plan to support the previous mos branch, like [fairseq-moe](https://github.com/facebookresearch/fairseq/tree/b5d89cddc9e4a0af831d2aafc1ba7dbf0f1b10d0/examples/moe_lm) on this library?
Contributor guide
Research direction
Review the linked fairseq-moe example at examples/moe_lm and compare its MoE support with fairseq2's current capabilities. The issue names no fairseq2 files, entry points, or tests, so first define the required scope and acceptance criteria before identifying implementation and test locations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100