FusedDispatch and FusedCombine use default config
Open
community-request
enhancement
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
Currently, FusedDispatch and FusedCombine use default config, which limit performance. Is it possible to allow user-defined configurations? For example, by reading a config file to set the chunk size accordingly.
[fused_a2a.py](https://github.com/NVIDIA/Megatron-LM/blob/b3426025fcf36f33d916590f2dee81e0e425ddd5/megatron/core/transformer/moe/fused_a2a.py#L113)
Contributor guide
Assessment
This issue has not been assessed yet.