kohya-ss / kohya-ss/sd-scripts

I got this error while learning 日本語(学習途中でこんなエラーが出ました

Open
#1,026 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

raceback (most recent call last):
File "D:\kohya\kohya_ss-22.0.0\train_network.py", line 1006, in
trainer.train(args)
File "D:\kohya\kohya_ss-22.0.0\train_network.py", line 819, in train
optimizer.step()
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\accelerate\optimizer.py", line 132, in step
self.scaler.step(self.optimizer, closure)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\cuda\amp\grad_scaler.py", line 374, in step
retval = self._maybe_opt_step(optimizer, optimizer_state, *args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\cuda\amp\grad_scaler.py", line 290, in _maybe_opt_step
retval = optimizer.step(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\accelerate\optimizer.py", line 185, in patched_step
return method(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\optim\lr_scheduler.py", line 69, in wrapper
return wrapped(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\optim\optimizer.py", line 280, in wrapper
out = func(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\bitsandbytes\optim\optimizer.py", line 265, in step
self.update_step(group, p, gindex, pindex)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\bitsandbytes\optim\optimizer.py", line 445, in update_step
config = self.get_config(gindex, pindex, group)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\bitsandbytes\optim\optimizer.py", line 280, in get_config
config["skip_zeros"] = self.args.skip_zeros
AttributeError: 'dict' object has no attribute 'skip_zeros'
steps: 13%|███████▋ | 257/2010 [01:11<08:06, 3.60it/s, loss=0.111]
Traceback (most recent call last):
File "C:\Users\souma\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\souma\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "D:\kohya\kohya_ss-22.0.0\venv\Scripts\accelerate.exe\__main__.py", line 7, in
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
args.func(args)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\accelerate\commands\launch.py", line 986, in launch_command
simple_launcher(args)
File "D:\kohya\kohya_ss-22.0.0\venv\lib\site-packages\accelerate\commands\launch.py", line 628, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['D:\\kohya\\kohya_ss-22.0.0\\venv\\Scripts\\python.exe', './train_network.py', '--enable_bucket', '--min_bucket_reso=256', '--max_bucket_reso=2048', '--pretrained_model_name_or_path=D:/SD20230727/stable-diffusion-webui/models/Stable-diffusion/train.ckpt', '--train_data_dir=D:/AI/dataset_768/Washio Sumi/img', '--resolution=768,768', '--output_dir=D:/AI/dataset_768/Washio Sumi/models', '--logging_dir=D:/AI/dataset_768/Washio Sumi/log', '--network_alpha=16', '--save_model_as=safetensors', '--network_module=networks.lora', '--text_encoder_lr=0.0009', '--unet_lr=0.00018', '--network_dim=32', '--output_name= Washio Sumi,1', '--lr_scheduler_num_cycles=3', '--no_half_vae', '--learning_rate=0.0004', '--lr_scheduler=cosine_with_restarts', '--train_batch_size=4', '--max_train_steps=2010', '--save_every_n_epochs=1', '--mixed_precision=fp16', '--save_precision=fp16', '--caption_extension=.txt', '--cache_latents', '--optimizer_type=AdamW8bit', '--max_data_loader_n_workers=0', '--bucket_reso_steps=64', '--xformers', '--bucket_no_upscale', '--noise_offset=0.0']' returned non-zero exit status 1.

During learning, an error like this occurred and learning stopped. 学習途中でこんなエラーが出て学習が止まりました
What kind of error is this? これはどういうエラーでしょうか?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with train_network.py around lines 819 and 1006, then trace the optimizer call through the dependency paths shown for accelerate, torch, and bitsandbytes. Reproduce the failure using the displayed training command and determine whether the optimizer configuration or dependency compatibility causes the AttributeError; done means a confirmed diagnosis or reproduction result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.