AI-Hypercomputer / AI-Hypercomputer/JetStream
MOE with JetStream
未關閉
- 主要語言
- Python
- 星號
- 457
- 分支
- 67
- PR 合併指標
- 30 天內沒有已合併 PR
描述
Could someone explain or point to a doc that explains how MOE is implemented on Jetstream? Specifically, the all-to-all communications, static vs dynamic, sparse matmuls.
I would like to understand how XLA compiles MOE.
貢獻指南
評估
這個 Issue 還沒有評估資料。