huggingface / huggingface/diffusers

Error in loading flux2 klein lora adapter

Open
#13,547 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

Hi,

I train my lora with DiffSynth repo for flux 2 klein model. When I want to inference with below code, I get warning and I think this problem is important.

inference code :


import torch
from diffusers import Flux2KleinPipeline
from glob import glob
from PIL import Image 

device = "cuda"
dtype = torch.bfloat16

model_path = "./weights/flux2-klein-9b"
pipe = Flux2KleinPipeline.from_pretrained(model_path, torch_dtype=dtype)
pipe.load_lora_weights(
    "./FLUX.2-klein-9B_lora",  
    weight_name="epoch-5.safetensors"  
)
pipe.fuse_lora(lora_scale=1.0)

pipe.enable_sequential_cpu_offload()  # save some VRAM by offloading the model to CPU

prompt = """You are an expert image adder."""

images_path = glob("./images/*")
for image_path in images_path:
    image  = Image.open(image_path)
    image_name = image_path.split('/')[-1]
    image = pipe(
        prompt=prompt,
        image = image,
        height=1024,
        width=1024,
        guidance_scale=1.0,
        num_inference_steps=4,
        generator=torch.Generator(device=device).manual_seed(0)
    ).images[0]
    image.save(f"./output/{image_name}")

warning

No LoRA keys associated to Flux2Transformer2DModel found with the prefix='transformer'. This is safe to ignore if LoRA state dict didn't originally have any Flux2Transformer2DModel related params. You can also try specifying prefix=None to resolve the warning.

Can someone help me ?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the warning with the provided Flux2KleinPipeline example, including load_lora_weights and fuse_lora, using the referenced DiffSynth-trained adapter. Then trace the pipeline's LoRA-loading entry points to determine whether the warning indicates a loading failure or an expected state-dict mismatch; done means the behavior is explained and the adapter loads or the limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.