kohya-ss / kohya-ss/sd-scripts
Error on training with DAdaptSGD
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
[Dataset 0]
caching latents.
100%|██████████████████████████████████████████████████████████████████████████████████| 29/29 [00:04<00:00, 6.48it/s]
create LoRA network. base dim (rank): 128, alpha: 64.0
neuron dropout: p=None, rank dropout: p=None, module dropout: p=None
create LoRA for Text Encoder: 72 modules.
create LoRA for U-Net: 192 modules.
enable LoRA for text encoder
enable LoRA for U-Net
preparing optimizer, data loader etc.
use D-Adaptation SGD optimizer | {}
running training / 学習開始
num train images * repeats / 学習画像の数×繰り返し回数: 870
num reg images / 正則化画像の数: 0
num batches per epoch / 1epochのバッチ数: 435
num epochs / epoch数: 10
batch size per device / バッチサイズ: 2
gradient accumulation steps / 勾配を合計するステップ数 = 1
total optimization steps / 学習ステップ数: 4350
steps: 0%| | 0/4350 [00:00
Contributor guide
No contributing guide indexed for this repository
Research direction
The report contains a training log for DAdaptSGD with loss becoming NaN during epoch 2, but names no source file or test. Start by reproducing the reported training run and trace the first non-finite loss through the optimizer and training entry point. Done means identifying the cause, preventing the NaN loss, and confirming that training completes with finite losses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100