deepspeedai / deepspeedai/DeepSpeedExamples
step2 bug fix for loss = nan when using BLOOM(which is left padding style)
@yaozhewei is already working on this.
Since Jun 16, 2023.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
When we use the bloom model to train the reward model, there may be a situation where the loss is always NaN. This is because the "end_ind" in the reward model is not correctly calculated, causing the "divergence_ind" to always be greater than "end_ind", so it is impossible to obtain the corresponding "chosen_reward" and "rejected_reward".
However, we know that since bloom uses left padding, the end position of "chosen" or "rejected" is always at the last index of the id. Therefore, we can simply set "end_ind = seq_len" in the forward function of the reward model, so that the end position can always be correctly obtained.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.