huggingface / huggingface/optimum-graphcore
Set LayerNorm's eps to a number that is larger than 6e-5
Open
- Dominant language
- Python
- Stars
- 87
- Forks
- 33
- PR merge metrics
- No merged PRs in 30d
Description
Currently, many LayerNorm's eps are smaller than 6.1e-5 (smallest fp16 value), which might cause underflow.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.