InternLM / InternLM/InternLM-XComposer
预训练阶段段loss下降后升高了，模型训崩了
Open
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
模拟xcomposer2的预训练,采用了相同的训练参数，也对vit采用了learning rate decay，训练过程中loss先慢慢下降，但后面升高了，升高之后发现模型训崩了，排除了训练数据问题，learning rate也调小了都不行，不知道是哪里的问题，放开vit训练久了之后就崩了？
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.