kohya-ss / kohya-ss/sd-scripts

how to use the trained lora safetensors in pipeline?

Open
#549 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

I used your great work to train a model, and the generated safetensors files works in WebUI;
however it could not be used in pure code with pipeline, such as
```
# in main
pipeline = StableDiffusionPipeline.from_ckpt('e:\\xxx\\trained\\stable-diffusion-v1-5-lxq-3.safetensors')
pipeline.to('cuda')
drawn = pipeline(
prompt='a girl with a cat',
height=1024,
width=1024,
num_inference_steps=20,
guidance_scale=5,
negative_prompt='lowres'
)
image = drawn.images[0]
image.show()
```

would result in error, like

```
global_step key not found in model
Traceback (most recent call last):
File "E:\xxx\taiyi\drawer\SafetensorsDrawTest.py", line 4, in
pipeline = StableDiffusionPipeline.from_ckpt(
File "E:\xxxr\venv\lib\site-packages\diffusers\loaders.py", line 1284, in from_ckpt
pipe = download_from_original_stable_diffusion_ckpt(
File "E:\xxx\venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 1177, in download_from_original_stable_diffusion_ckpt
converted_unet_checkpoint = convert_ldm_unet_checkpoint(
File "E:\xxx\venv\lib\site-packages\diffusers\pipelines\stable_diffusion\convert_from_ckpt.py", line 376, in convert_ldm_unet_checkpoint
new_checkpoint["time_embedding.linear_1.weight"] = unet_state_dict["time_embed.0.weight"]
KeyError: 'time_embed.0.weight'
```

Could you please give some advice?
Thx.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reported StableDiffusionPipeline.from_ckpt call and the traceback through download_from_original_stable_diffusion_ckpt and convert_ldm_unet_checkpoint. Determine whether the trained safetensors format is supported by this pipeline, then document the supported loading procedure or the incompatibility and its workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.