`GradScaler` and mixed-precision training
Open
- Dominant language
- C#
- Stars
- 1.9k
- Forks
- 228
- PR merge metrics
- No merged PRs in 30d
Description
https://pytorch.org/docs/stable/notes/amp_examples.html
Currently, `bfloat16` works well without grad scaling. But to use `fp16` and `fp8` (`fp8` - in the future, when the support for Hopper/40XX GPUs lands) one needs to scale gradients.
Contributor guide
Assessment
This issue has not been assessed yet.