modelscope / modelscope/DiffSynth-Studio

klein lora: ERROR lora diffusion_model.double_blocks.4.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184

Open
#1,267 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

use convert_script:
from safetensors import safe_open
from safetensors.torch import save_file

def convert_flux2_lora_keys(input_file, output_file):
new_data = {}
metadata = {}

with safe_open(input_file, framework="pt") as f:
    if hasattr(f, 'metadata'):
        metadata = f.metadata() or {}
    
    print(f"Converting {len(f.keys())} keys...\n")
    print("=" * 80)
    
    for key in f.keys():
        original_key = key
        new_key = key.replace('.default', '')  # Remove .default suffix
        
        # === SINGLE BLOCKS MAPPING ===
        if new_key.startswith('single_transformer_blocks.'):
            # Replace block name
            new_key = new_key.replace('single_transformer_blocks.', 'single_blocks.')
            
            # Map layer names:
            # to_qkv_mlp_proj -> linear1
            # to_out -> linear2
            new_key = new_key.replace('.attn.to_qkv_mlp_proj.', '.linear1.')
            new_key = new_key.replace('.attn.to_out.', '.linear2.')
        
        # === DOUBLE BLOCKS MAPPING ===
        elif new_key.startswith('transformer_blocks.'):
            new_key = new_key.replace('transformer_blocks.', 'double_blocks.')
            
            # Image attention mappings - keep Q, K, V separate
            if '.attn.to_q.' in new_key:
                new_key = new_key.replace('.attn.to_q.', '.img_attn.qkv.')
            elif '.attn.to_k.' in new_key:
                new_key = new_key.replace('.attn.to_k.', '.img_attn.qkv.')
            elif '.attn.to_v.' in new_key:
                new_key = new_key.replace('.attn.to_v.', '.img_attn.qkv.')
            elif '.attn.to_out.0.' in new_key:
                new_key = new_key.replace('.attn.to_out.0.', '.img_attn.proj.')
            
            # Text attention mappings - keep Q, K, V separate
            elif '.attn.add_q_proj.' in new_key:
                new_key = new_key.replace('.attn.add_q_proj.', '.txt_attn.qkv.')
            elif '.attn.add_k_proj.' in new_key:
                new_key = new_key.replace('.attn.add_k_proj.', '.txt_attn.qkv.')
            elif '.attn.add_v_proj.' in new_key:
                new_key = new_key.replace('.attn.add_v_proj.', '.txt_attn.qkv.')
            elif '.attn.to_add_out.' in new_key:
                new_key = new_key.replace('.attn.to_add_out.', '.txt_attn.proj.')
            
            # Feed-forward mappings
            # ff -> img_mlp
            # ff_context -> txt_mlp
            elif '.ff.linear_in.' in new_key:
                new_key = new_key.replace('.ff.linear_in.', '.img_mlp.0.')
            elif '.ff.linear_out.' in new_key:
                new_key = new_key.replace('.ff.linear_out.', '.img_mlp.2.')
            elif '.ff_context.linear_in.' in new_key:
                new_key = new_key.replace('.ff_context.linear_in.', '.txt_mlp.0.')
            elif '.ff_context.linear_out.' in new_key:
                new_key = new_key.replace('.ff_context.linear_out.', '.txt_mlp.2.')
        
        # Add diffusion_model prefix
        new_key = f"diffusion_model.{new_key}"
        
        print(f"{original_key}")
        print(f"  -> {new_key}")
        
        new_data[new_key] = f.get_tensor(original_key)

print("\n" + "=" * 80)
print(f"Successfully converted {len(new_data)} keys")
save_file(new_data, output_file, metadata=metadata)
print(f"✓ Saved to: {output_file}")
print(f"\nPlace this file in: ComfyUI/models/loras/")

if name == "main":
import sys

if len(sys.argv) > 1:
    input_file = sys.argv[1]
    output_file = sys.argv[2] if len(sys.argv) > 2 else input_file.replace('.safetensors', '_comfy.safetensors')
else:
    input_file = "old_lora.safetensors"
    output_file = "new_lora.safetensors"

convert_flux2_lora_keys(input_file, output_file)

found:

ERROR lora diffusion_model.double_blocks.4.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.4.img_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.3.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.3.img_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.2.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.2.img_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.1.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.1.img_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.0.txt_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184
ERROR lora diffusion_model.double_blocks.0.img_attn.qkv.weight shape '[9216, 3072]' is invalid for input of size 9437184

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the inline Python entry point convert_flux2_lora_keys and reproduce the reported conversion and loading errors using the shown script. Inspect the generated double_blocks.*.txt_attn.qkv and img_attn.qkv tensors against the reported shapes. Done means the converted LoRA loads without the invalid-shape errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.