AnswerDotAI / AnswerDotAI/ModernBERT
grad_norm Nan question
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
{'loss': 0.2051, 'grad_norm': nan, 'learning_rate': 9.774074342405685e-06, 'epoch': 0.53}
{'loss': 0.0, 'grad_norm': nan, 'learning_rate': 9.773696745450087e-06, 'epoch': 0.53}
{'loss': 0.0, 'grad_norm': nan, 'learning_rate': 9.773318840518906e-06, 'epoch': 0.53}
How to adjust the gradient such as Nan loss and 0 when training to epoch=0.53?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.