finetune的时候为何没有把<|Human|>的loss给mask掉?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
with open(os.path.join(self.data_dir, f'{self.data_type}.jsonl'), 'r') as f:
for line in f:
sample = json.loads(line)
chat = sample['chat']
num_turns = int(sample['num_turns'])
meta_instruction = sample['meta_instruction']
instruction_ids = self.tokenizer.encode(meta_instruction)
assert isinstance(instruction_ids, list) and len(instruction_ids) > 0
input_ids = copy.deepcopy(instruction_ids)
no_loss_spans = [(0, len(instruction_ids))]
for i in range(num_turns):
cur_turn_ids = []
cur_no_loss_spans = []
cur_turn = chat[f'turn_{i+1}']
for key, value in cur_turn.items():
cur_ids = self.tokenizer.encode(value)
if key == 'Tool Responses':
# The format tokens (<|Results|>:...<eor>\n) should have losses.
cur_no_loss_spans.append((len(input_ids + cur_turn_ids) + 5, len(input_ids + cur_turn_ids + cur_ids) - 2))
assert isinstance(cur_ids, list) and len(cur_ids) > 0
cur_turn_ids.extend(cur_ids)
if len(input_ids + cur_turn_ids) > 2048:
break
input_ids.extend(cur_turn_ids)
no_loss_spans.extend(cur_no_loss_spans)
if len(input_ids) == len(instruction_ids):
continue
assert len(input_ids) > 0 and len(input_ids) <= 2048
self.data.append(input_ids)
self.no_loss_spans.append(no_loss_spans)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue shows a Python JSONL-loading loop that builds input_ids and no_loss_spans; start by locating where self.no_loss_spans is consumed during fine-tuning. Trace how the and Tool Responses sections are represented, then verify that the loss mask matches the intended training spans. Done means the masking behavior is clarified or corrected and covered by the project's existing validation path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100