InternLM / InternLM/InternLM-XComposer
About PLora
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
If I want to turn off parameters in the Plora section during training, which layers of the model do I need to turn off parameter iterations?
"model.layers.0.attention.wo.Plora_A.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention.wo.Plora_B.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention.wo.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention.wqkv.Plora_A.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention.wqkv.Plora_B.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention.wqkv.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.attention_norm.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w1.Plora_A.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w1.Plora_B.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w1.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w2.Plora_A.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w2.Plora_B.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w2.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w3.Plora_A.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w3.Plora_B.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.feed_forward.w3.weight": "pytorch_model-00001-of-00002.bin",
"model.layers.0.ffn_norm.weight": "pytorch_model-00001-of-00002.bin",
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.