huggingface / huggingface/diffusers

Getting problems saving text_encoder_ti when using train_dreambooth_lora_sd15_advanced

Open
#8,909 3 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

Hi,

I encountered an issue while training Stable Diffusion 1.5 using the script `examples/advanced_diffusion_training/train_dreambooth_lora_sd15_advanced.py`.

The problem occurs when trying to save the text_encoder while also using textual inversion.

The following line of code seems to be causing the issue:

```python
embedding_handler.save_embeddings(f"{output_dir}/{args.output_dir}_emb.safetensors")
```

I discovered that removing {args.output_dir} from the path resolves the problem:

```
embedding_handler.save_embeddings(f"{output_dir}/ti_emb.safetensors")
```

Are there any alternative solutions to this issue? Any insights would be appreciated.

image

image

### Reproduction

Use the script train_dreambooth_lora_sd15_advanced.py
Enable textual inversion
Attempt to save the text_encoder

### Logs

_No response_

### System Info

- 🤗 Diffusers version: 0.30.0.dev0
- Platform: Linux-5.15.0-106-generic-x86_64-with-glibc2.35
- Running on a notebook?: No
- Running on Google Colab?: No
- Python version: 3.10.12
- PyTorch version (GPU?): 2.3.1+cu121 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.23.4
- Transformers version: 4.42.3
- Accelerate version: 0.31.0
- PEFT version: 0.11.1
- Bitsandbytes version: not installed
- Safetensors version: 0.4.3
- xFormers version: 0.0.27
- Accelerator: NVIDIA A40, 46068 MiB
NVIDIA A40, 46068 MiB
NVIDIA A40, 46068 MiB
NVIDIA A40, 46068 MiB VRAM
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with examples/advanced_diffusion_training/train_dreambooth_lora_sd15_advanced.py and reproduce the run with textual inversion enabled and text_encoder saving enabled. Inspect the embedding_handler.save_embeddings path shown in the issue and verify that the text encoder and textual-inversion embeddings save successfully to the intended output directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.