sgl-project / sgl-project/SpecForge

[Bug] Issue with the loss mask of eagle3 in data/parse.py

Open Beginner friendly
#885 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
347
Avg merge
4d 1h
Merged PRs (30d)
41

Description

Checklist
  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.
Describe the bug

In the eagle3 training pipeline, target_head.preprocess shifts BOTH input_ids and target LEFT by one, so at chain step 0 position p carries:
input_embed = embed(orig_ids[p + 1])
aux = aux[p]
label = argmax(lm_head(orig_hidden[p + 1])) = orig[p + 2]
But loss_mask is NOT shifted — it still marks orig[p] ∈ assistant span. As a result, chain step 0 at position T_start - 1 (which predicts the SECOND assistant token, orig[T_start + 1]) is masked out entirely, even though it's the position that runtime uses in round 1 spec_1. Extending loss_mask backward by 1 recovers loss at that boundary position so the draft is trained to predict "accept vs reject" from the end-of-prompt aux.

Need to change loss_mask[actual_start:actual_end] = 1 to loss_mask[max(0, actual_start - 1):actual_end] = 1 in the parse function of GeneralParser.

Reproduction

Train command is: specforge train --config xxx
The model is minicpm5-1B

Environment

h20

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in data/parse.py at GeneralParser.parse and inspect how target_head.preprocess shifts input_ids and targets relative to loss_mask. Run the reported specforge train --config xxx reproduction with minicpm5-1B, then verify the loss mask includes the shifted boundary position at the start of the assistant span.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.