facebookresearch / facebookresearch/fairscale
[Feature] Re-Implement MEFP16 by replacing the fused Adam CUDA kernel with a Triton based one
Open
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 293
- PR merge metrics
- No merged PRs in 30d
Description
see here: https://github.com/facebookresearch/fairscale/tree/main/fairscale/clib/fused_adam_cuda
This has been an issue in terms of building it from source. If it can be replaced with a triton one with similar to better performance, maybe we should do that to ease the fairscale installation process.
Contributor guide
Assessment
This issue has not been assessed yet.