deepseek-ai / deepseek-ai/DeepSeek-Math
关于sft阶段中数据拼接的问题
Open
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 592
- PR merge metrics
- No merged PRs in 30d
Description
您好,论文3.2中有提到将训练数据随机拼接到4k token的长度,请问是指将sft数据拼接成(q0,a0,q1,a1,...)的形式后只计算answer部分的loss吗?
非常感谢大佬们的工作~
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue asks how section 3.2 of the paper handles randomly concatenated SFT data and answer-only loss, but names no repository file or test. Start by reading section 3.2 and locating the repository's SFT training implementation. Done means documenting a maintainer-confirmed explanation of the data format and loss calculation.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100