sdxl_train_network.py Error

sdxl_train_network.pyでnetwork_train_unet_onlyが有効かつBlock weightを使用するとエラーで学習ができません。AdamWでは以下のエラーが発生し、ほかのOptimizerでもlist index out of rangeなどのエラーで動作しません。
commit hashは`89aae3e04fe6aa11c3da76b9b48eed16be310b14`です。
```
Traceback (most recent call last):
File "G:\sd-scripts\sdxl_train_network.py", line 174, in
trainer.train(args)
File "G:\sd-scripts\train_network.py", line 324, in train
optimizer_name, optimizer_args, optimizer = train_util.get_optimizer(args, trainable_params)
File "G:\sd-scripts\library\train_util.py", line 3376, in get_optimizer
optimizer = optimizer_class(trainable_params, lr=lr, **optimizer_kwargs)
File "G:\sd-scripts\venv\lib\site-packages\torch\optim\adamw.py", line 50, in __init__
super().__init__(params, defaults)
File "G:\sd-scripts\venv\lib\site-packages\torch\optim\optimizer.py", line 187, in __init__
raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list
```

コマンドは以下の通りです。
```
accelerate launch --num_cpu_threads_per_process 4 sdxl_train_network.py --pretrained_model_name_or_path "G:\sd\models\Stable-diffusion\sd_xl_base_1.0.safetensors" --train_data_dir "Dataset path" --output_dir "Output path" --network_module "networks.lora" --network_args "conv_dim=64" "conv_alpha=12" "down_lr_weight=0,0,0,0,0,0,0,0,0,0,0,0" "mid_lr_weight=0" "up_lr_weight=0,1,1,1,1,1,1,1,0,0,0,0" --xformers --gradient_checkpointing --persistent_data_loader_workers --cache_latents --cache_latents_to_disk --max_data_loader_n_workers 1 --enable_bucket --save_model_as "safetensors" --lr_scheduler_num_cycles 4 --mixed_precision "fp16" --learning_rate 0.0001 --resolution 512 --train_batch_size 1 --max_train_epochs 10 --network_dim 64 --network_alpha 12 --optimizer_type "AdamW" --lr_warmup_steps 250 --output_name "mika_v16_xl" --clip_skip 2 --save_precision "fp16" --lr_scheduler "cosine_with_restarts" --min_bucket_reso 320 --max_bucket_reso 1536 --caption_extension ".txt" --seed 42 --network_train_unet_only --no_half_vae
```

Start with sdxl_train_network.py and follow train_network.py into library/train_util.py, especially get_optimizer and the trainable_params passed to it. Reproduce the command at commit 89aae3e04fe6aa11c3da76b9b48eed16be310b14 with network_train_unet_only and the block weights, then confirm training no longer reaches an empty optimizer parameter list or related index errors.

Written by the indexing model from the issue text.