kohya-ss / kohya-ss/sd-scripts
Question about V-Prediction in SDXL Finetuning
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
It's just my one-sided doubts, about the implement of the v-prediction.
In sdxl training, the source code implements v-prediction by:
```
def add_v_prediction_like_loss(loss, timesteps, noise_scheduler, v_pred_like_loss):
scale = get_snr_scale(timesteps, noise_scheduler)
# print(f"add v-prediction like loss: {v_pred_like_loss}, scale: {scale}, loss: {loss}, time: {timesteps}")
loss = loss + loss / scale * v_pred_like_loss
return loss
```
which is mathematically equivalent to:
L:=L+snr\*L\*w,
where w=v_pred_like_loss, and snr=scale,
while the paper suggests:
L:=snr\*L.
So, is the source adds additional v-pred like loss rather than scaling it? Why are the implementation and paper different?
I'm not a mathematician, and maybe I'm short-sighted. Hope someone can answer my doubts :D
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the add_v_prediction_like_loss function shown in the issue and trace its callers in the SDXL training code. Compare get_snr_scale and the implementation with the referenced paper to determine whether the behavior is intentional. Done would be a maintainer-confirmed explanation or clarified documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100