huggingface / huggingface/diffusers

[Request] Compatibility of textual inversion between `SD 1.5` and `SD 2.1`

Open
#10,373 2 comments 0 reactions 0 assignees View on GitHub
contributions-welcome stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

**Is your feature request related to a problem? Please describe.**
Textual inversions trained on different versions of SD, such as SD 1.5 and SD 2.1, are not compatible.
Related to previously submitted issue #4030

**Describe the solution you'd like.**
To make textual inversion, which is compatible only with SD 1.5, also compatible with SD 2.1

**Describe alternatives you've considered.**
I was trying to replace [this part](https://github.com/huggingface/diffusers/blob/825979ddc3d03462287f1f5439e89ccac8cc71e9/src/diffusers/loaders/textual_inversion.py#L406C13-L409C14) with a process that resizes to the expected shape.

**Additional context.**
Less important, error messages are not displayed correctly.

```python
import torch
from diffusers import StableDiffusionPipeline
from huggingface_hub import hf_hub_download

pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16)

path = hf_hub_download(repo_id="gsdf/EasyNegative", filename="EasyNegative.safetensors", repo_type="dataset")

pipe.load_textual_inversion(path,token="EasyNegative")

# ValueError: Loaded embeddings are of incorrect shape. Expected each textual inversion embedding to be of shape {input_embeddings.shape[-1]}, but are {embeddings.shape[-1]}
```

cc: @sayakpaul

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.