sgl-project / sgl-project/SpecForge
[Bug] Issue with the loss mask of eagle3 in data/parse.py
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 347
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 41
Description
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
- 5. Please use English, otherwise it will be closed.
Describe the bug
In the eagle3 training pipeline, target_head.preprocess shifts BOTH input_ids and target LEFT by one, so at chain step 0 position p carries:
input_embed = embed(orig_ids[p + 1])
aux = aux[p]
label = argmax(lm_head(orig_hidden[p + 1])) = orig[p + 2]
But loss_mask is NOT shifted — it still marks orig[p] ∈ assistant span. As a result, chain step 0 at position T_start - 1 (which predicts the SECOND assistant token, orig[T_start + 1]) is masked out entirely, even though it's the position that runtime uses in round 1 spec_1. Extending loss_mask backward by 1 recovers loss at that boundary position so the draft is trained to predict "accept vs reject" from the end-of-prompt aux.
Need to change loss_mask[actual_start:actual_end] = 1 to loss_mask[max(0, actual_start - 1):actual_end] = 1 in the parse function of GeneralParser.
Reproduction
Train command is: specforge train --config xxx
The model is minicpm5-1B
Environment
h20
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in data/parse.py at GeneralParser.parse and inspect how target_head.preprocess shifts input_ids and targets relative to loss_mask. Run the reported specforge train --config xxx reproduction with minicpm5-1B, then verify the loss mask includes the shifted boundary position at the start of the assistant span.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100