huggingface / huggingface/course

Use DataCollatorForCompletionOnlyLM in order to train LLM to follow instructions

Open
#845 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

In Chapter 11 of the course, in the file **FineTuning with SFTTrainer** (`3.mdx`), you explain how to fine-tune a DeepSeek model with `SFTTrainer` on an instruction dataset.

Why don't you use the `DataCollatorForCompletionOnlyLM` data collator in the `SFTTrainer` to avoid computing gradients and back-propagating on the user question tokens? The default data collator in the `SFTTrainer` is `DataCollatorForLanguageModeling`, which means that in your example the LLM will also learn to complete the user query, which is not intended as far as I understand.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.