kohya-ss / kohya-ss/sd-scripts
When using masked loss, the console output is confusing as hell.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
When using masked loss, the console output is confusing as hell.
The simplest dataset config:
```
[[datasets]]
[[datasets.subsets]]
conditioning_data_dir = 'E:\\tmp\\masks'
image_dir = 'E:\\tmp\\train_dataset\\1_foobar_xl'
```
All other settings are set in the main .toml file (including enable_bucket = true, etc.) [Here](https://pastebin.com/8ix3Gk5i)
In the console, we see something like this:
```
INFO 63 train images with repeating. train_util.py:2010
INFO 0 reg images. train_util.py:2013
WARNING no regularization images / 正則化画像が見つかりませんでした train_util.py:2018
INFO [Dataset 0] config_util.py:567
batch_size: 4
resolution: (1024, 1024)
enable_bucket: False
network_multiplier: 1.0
2024-11-09 23:41:01 INFO [Dataset 0] config_util.py:573
2024-11-09 23:41:06 INFO loading image sizes. train_util.py:923
100%|████████████████████████████████████████████████████████████████████████████████| 63/63 [00:00<00:00, 9636.10it/s]
INFO make buckets train_util.py:946
WARNING min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is train_util.py:963
set, because bucket reso is defined by image size automatically /
bucket_no_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計
算されるため、min_bucket_resoとmax_bucket_resoは無視されます
INFO number of images (including repeats) / train_util.py:992
各bucketの画像枚数(繰り返し回数を含む)
INFO bucket 0: resolution (704, 1344), count: 2 train_util.py:997
INFO bucket 1: resolution (768, 1280), count: 3 train_util.py:997
INFO bucket 2: resolution (832, 1216), count: 22 train_util.py:997
INFO bucket 3: resolution (896, 1152), count: 17 train_util.py:997
INFO bucket 4: resolution (960, 1088), count: 5 train_util.py:997
INFO bucket 5: resolution (1024, 1024), count: 9 train_util.py:997
INFO bucket 6: resolution (1088, 960), count: 2 train_util.py:997
INFO bucket 7: resolution (1152, 896), count: 2 train_util.py:997
INFO bucket 8: resolution (1216, 832), count: 1 train_util.py:997
INFO mean ar error (without repeats): 0.0
```
**enable_bucket: False**? No subsets? And then **make buckets**?
There is no dataset and no captions information in the metadata of the final file either.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the dataset and bucket logging in train_util.py around lines 923, 946, 963, 992, and 997, then compare it with the Dataset output in config_util.py around lines 567 and 573. Reproduce the provided masked-loss configuration and determine how the console should distinguish dataset settings, subsets, and bucket processing. Done means the output is consistent with the configuration and the final file reports the relevant dataset and caption metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100