kohya-ss / kohya-ss/sd-scripts

アップデート後にdadaptation入力値に対してランタイムエラーが発生します

Open
#555 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

```
RuntimeError: Setting different lr values in different parameter groups is only supported for values of 0
steps: 0%| | 1/1750 [00:28<13:57:53, 28.74s/it, loss=0.0571]
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "D:\AI\kohya_ss\venv\Scripts\accelerate.exe\__main__.py", line 7, in
File "D:\AI\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 45, in main
args.func(args)
File "D:\AI\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 923, in launch_command
simple_launcher(args)
File "D:\AI\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 579, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['D:\\AI\\kohya_ss\\venv\\Scripts\\python.exe', 'train_network.py', '--enable_bucket', '--pretrained_model_name_or_path=D:/AI/stable-diffusion-webui/models/Stable-diffusion/train.ckpt', '--train_data_dir=D:\\AI\\dataset\\TEST\\img', '--resolution=768,768', '--output_dir=D:\\AI\\dataset\\TEST\\model', '--logging_dir=D:\\AI\\dataset\\TEST\\log', '--network_alpha=32', '--save_model_as=safetensors', '--network_module=networks.lora', '--text_encoder_lr=0.5', '--unet_lr=1.0', '--network_dim=32', '--output_name=TEST_v1', '--lr_scheduler_num_cycles=4', '--learning_rate=1.0', '--lr_scheduler=constant_with_warmup', '--lr_warmup_steps=175', '--train_batch_size=4', '--max_train_steps=1750', '--save_every_n_epochs=1', '--mixed_precision=fp16', '--save_precision=fp16', '--seed=1234', '--caption_extension=.txt', '--cache_latents', '--optimizer_type=DAdaptation', '--max_token_length=225', '--clip_skip=2', '--keep_tokens=1', '--bucket_reso_steps=64', '--xformers', '--bucket_no_upscale']' returned non-zero exit status 1.
```

アップデート以前は

Learning rate 1.0
Text Encoder learning rate : 0.5
Unet learning rate : 1.0

でLoRAを作ってきましたが、今回のdadaptationアップデート以降は異なる値で入力して学習を開始する場合、ランタイムエラーが発生します。

`RuntimeError: Setting different lr values in different parameter groups is only supported for values of 0`

すべてのdadaptation関連オプティマイザーで同じ現象ですが、これは意図された現象なのか知りたいです。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the command shown in the issue with train_network.py, using DAdaptation and different text encoder and U-Net learning rates. Inspect where the optimizer and parameter groups are configured, then compare the behavior before and after the DAdaptation update. Done means determining whether the restriction is intentional and documenting or correcting the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.