deepseek-ai / deepseek-ai/DeepSeek-Math

关于sft阶段中数据拼接的问题

Open
#23 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
592
PR merge metrics
No merged PRs in 30d

Description

您好,论文3.2中有提到将训练数据随机拼接到4k token的长度,请问是指将sft数据拼接成(q0,a0,q1,a1,...)的形式后只计算answer部分的loss吗?
非常感谢大佬们的工作~

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue asks how section 3.2 of the paper handles randomly concatenated SFT data and answer-only loss, but names no repository file or test. Start by reading section 3.2 and locating the repository's SFT training implementation. Done means documenting a maintainer-confirmed explanation of the data format and loss calculation.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.