kohya-ss / kohya-ss/sd-scripts

When I try to train Lora on my MacBook Pro, keep getting error

Open
#479 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

My MacBook Pro has 16GB VRAM but I keep getting error when I tried to try Lora on it, anyone can help?

running training / 学習開始
num train images * repeats / 学習画像の数×繰り返し回数: 2000
num reg images / 正則化画像の数: 0
num batches per epoch / 1epochのバッチ数: 2000
num epochs / epoch数: 1
batch size per device / バッチサイズ: 1
gradient accumulation steps / 勾配を合計するステップ数 = 1
total optimization steps / 学習ステップ数: 2000
steps: 0%| | 0/2000 [00:00
train(args)
File "/Users/parkson/kohya_ss/train_network.py", line 647, in train
accelerator.clip_grad_norm_(params_to_clip, args.max_grad_norm)
File "/Users/parkson/kohya_ss/venv/lib/python3.10/site-packages/accelerate/accelerator.py", line 1373, in clip_grad_norm_
self.unscale_gradients()
File "/Users/parkson/kohya_ss/venv/lib/python3.10/site-packages/accelerate/accelerator.py", line 1336, in unscale_gradients
self.scaler.unscale_(opt)
AttributeError: 'NoneType' object has no attribute 'unscale_'
steps: 0%| | 0/2000 [14:54
sys.exit(main())
File "/Users/parkson/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 45, in main
args.func(args)
File "/Users/parkson/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1104, in launch_command
simple_launcher(args)
File "/Users/parkson/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 567, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/Users/parkson/kohya_ss/venv/bin/python', 'train_network.py', '--pretrained_model_name_or_path=runwayml/stable-diffusion-v1-5', '--train_data_dir=/Users/parkson/Desktop/TEMP/JulianaChoi_Lora/Image', '--resolution=512,512', '--output_dir=/Users/parkson/Desktop/TEMP/JulianaChoi_Lora/Model', '--logging_dir=/Users/parkson/Desktop/TEMP/JulianaChoi_Lora/Log', '--network_alpha=128', '--save_model_as=safetensors', '--network_module=networks.lora', '--text_encoder_lr=5e-5', '--unet_lr=0.0001', '--network_dim=128', '--output_name=JulianaChoi', '--lr_scheduler_num_cycles=1', '--learning_rate=0.0001', '--lr_scheduler=constant', '--train_batch_size=1', '--max_train_steps=2000', '--save_every_n_epochs=1', '--mixed_precision=bf16', '--save_precision=bf16', '--seed=1234', '--caption_extension=.txt', '--cache_latents', '--optimizer_type=AdamW8bit', '--max_data_loader_n_workers=1', '--clip_skip=2', '--bucket_reso_steps=64', '--mem_eff_attn', '--bucket_no_upscale']' returned non-zero exit status 1.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with train_network.py around line 647 and compare the failing accelerator.clip_grad_norm_ call with the traceback in accelerate/accelerator.py. Reproduce the reported MacBook Pro command and determine the compatibility or configuration cause of the None scaler; done means LoRA training proceeds past the first optimization step.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.