Performance issues for SFT on Qwen 30B-a3b
Open
Performance
research
t-mcore
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Performance of SFT for Qwen 30B-a3b is ~30% lower than Qwen 14B probably because of it being an MoE model. Can we improve performance of SFT Qwen 30B? This is for megatron backend.
Contributor guide
Assessment
This issue has not been assessed yet.