[QUESTION] How to get delta weights after finetuning?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
thanks to the awesome work!
In train_mem.py, all parameters have been finetuned used Trainer from transformers instead of using LORA, but delta weights are released only.
So can you tell me your method to get delta weights after full parameters finetuning? thanks a lot
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading train_mem.py and its Trainer configuration to understand how full-parameter fine-tuning differs from the released delta weights. Check the surrounding training documentation for an existing explanation; done means documenting a reproducible method for obtaining delta weights after full-parameter fine-tuning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100