deepspeedai / deepspeedai/DeepSpeedExamples
Using LLaMA in reward model training
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
Hi,
I have encounter the TypeError: LlamaModel.forward() got an unexpected keyword argument 'head_mask' error when training the LLaMA-7B model in step 2 reward model training.
I was wondering if the head_mask is used at all in training the reward model?
Also, is there a quick fix for this error?
Many thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the LLaMA-7B step 2 reward model training path and the call that reaches LlamaModel.forward. Reproduce the head_mask TypeError and determine whether head_mask is required for reward model training. Done means the training path no longer passes an unsupported argument, or the issue documents why it is needed and how to handle it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100