kohya-ss / kohya-ss/sd-scripts

Question about V-Prediction in SDXL Finetuning

Open
#1,163 1 comment 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

It's just my one-sided doubts, about the implement of the v-prediction.
In sdxl training, the source code implements v-prediction by:
```
def add_v_prediction_like_loss(loss, timesteps, noise_scheduler, v_pred_like_loss):
scale = get_snr_scale(timesteps, noise_scheduler)
# print(f"add v-prediction like loss: {v_pred_like_loss}, scale: {scale}, loss: {loss}, time: {timesteps}")
loss = loss + loss / scale * v_pred_like_loss
return loss
```

which is mathematically equivalent to:
L:=L+snr\*L\*w,
where w=v_pred_like_loss, and snr=scale,
while the paper suggests:
L:=snr\*L.

So, is the source adds additional v-pred like loss rather than scaling it? Why are the implementation and paper different?
I'm not a mathematician, and maybe I'm short-sighted. Hope someone can answer my doubts :D

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the add_v_prediction_like_loss function shown in the issue and trace its callers in the SDXL training code. Compare get_snr_scale and the implementation with the referenced paper to determine whether the behavior is intentional. Done would be a maintainer-confirmed explanation or clarified documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.