InternLM / InternLM/InternEvo

[Feature] Partially frozen model support

Open
#11 2 comments 0 reactions 1 assignee Claimed by @SolenoidWGT View on GitHub
enhancement
Dominant language
Python
Stars
421
Forks
67
PR merge metrics
No merged PRs in 30d

Description

### Describe the feature

Current implementation does not support partially training, i.e., part of the model parameters are frozen.
If I understand it correctly, the assertion around line 584 in hybrid_zero_optim.py suggests that all parameters should be involved in the training.
image

Very looking forward this feature being implemented as there are many scenarios where users only want to finetune parts of the model.

Is there any plan on this feature?

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.