kohya-ss / kohya-ss/sd-scripts
sdxl_merge_lora.py does not work properly
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
In networks/sdxl_merge_lora.py line 80
`if module_name not in name_to_module:`
The **module_name** is always not in **name_to_module**
because the different name between unet and diffuser format
**name_to_module** contains something like **lora_unet_input_blocks_1_0_in_layers_2**
**module_name** always like **lora_unet_down_blocks_0_resnets_0_conv1**
It should have a convertion or mapping between different name.
I made a .json file for mapping but I think it might not the best solution.
[diffusers_keys_mapping.json](https://github.com/kohya-ss/sd-scripts/files/13467571/diffusers_keys_mapping.json)
My workaround:
```
...
diffusers_keys = {}
script_dir = os.path.dirname(__file__)
mapping_file_path = os.path.join(script_dir, "diffusers_keys_mapping.json")
# open diffusers_keys.json file
with open(mapping_file_path, "r") as f:
diffusers_keys = json.load(f)
f.close()
for model, ratio in zip(models, ratios):
print(f"loading: {model}")
lora_sd, _ = load_state_dict(model, merge_dtype)
print(f"merging...")
for key in tqdm(lora_sd.keys()):
if "lora_down" in key:
up_key = key.replace("lora_down", "lora_up")
alpha_key = key[: key.index("lora_down")] + "alpha"
# find original module for this lora
module_name = ".".join(key.split(".")[:-2]) # remove trailing ".lora_down.weight"
patch_module_name = diffusers_keys[module_name]
# print(f'debug key:{key}')
print(f'debug patch_module_name:{patch_module_name}')
if patch_module_name not in name_to_module:
print(f"no module found for LoRA weight: {key}")
...
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at networks/sdxl_merge_lora.py line 80 and compare the unet and diffuser-format names described in the issue. Review the proposed diffusers_keys_mapping.json workaround, then verify that SDXL LoRA modules are found during merging without the reported name mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100