InternLM / InternLM/InternLM-XComposer
layer-wise learning rate
Open
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
请教一个问题，layer-wise learning rate 比如同一层的参数有Q、K、V、MLP等参数，同一层的参数学习率相同，还是按照从上到下的顺序，同一层的lr也会衰减，同一层的学习率也不同？
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.