huggingface / huggingface/course
Use DataCollatorForCompletionOnlyLM in order to train LLM to follow instructions
Open
- Dominant language
- MDX
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
In Chapter 11 of the course, in the file **FineTuning with SFTTrainer** (`3.mdx`), you explain how to fine-tune a DeepSeek model with `SFTTrainer` on an instruction dataset.
Why don't you use the `DataCollatorForCompletionOnlyLM` data collator in the `SFTTrainer` to avoid computing gradients and back-propagating on the user question tokens? The default data collator in the `SFTTrainer` is `DataCollatorForLanguageModeling`, which means that in your example the LLM will also learn to complete the user query, which is not intended as far as I understand.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.