[Feature] Partially frozen model support
Open
enhancement
- Dominant language
- Python
- Stars
- 421
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
### Describe the feature
Current implementation does not support partially training, i.e., part of the model parameters are frozen.
If I understand it correctly, the assertion around line 584 in hybrid_zero_optim.py suggests that all parameters should be involved in the training.
Very looking forward this feature being implemented as there are many scenarios where users only want to finetune parts of the model.
Is there any plan on this feature?
### Will you implement it?
- [ ] I would like to implement this feature and create a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.