huggingface / huggingface/diffusers

Train_controlnet_sdxl.py and tensorboard log images

Open
#11,470 1 comment 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

When logging validation images on tensorboard, one tile is always created, even if there are several validation projects and images. I suggest making changes to the code:
```
for tracker in accelerator.trackers:
if tracker.name == "tensorboard":
i=0
for log in image_logs:
images = log["images"]
validation_prompt = log["validation_prompt"]
validation_image = log["validation_image"]

formatted_images = [np.asarray(validation_image)]

for image in images:
formatted_images.append(np.asarray(image))

formatted_images = np.stack(formatted_images)

tag = f"{validation_prompt}_{i}"
tracker.writer.add_images(tag, formatted_images, step, dataformats="NHWC")
i=i+1
tracker.writer.flush()
elif tracker.name == "wandb":
```

### Reproduction

Start Train_controlnet_sdxl.py with params:

```
accelerate
launch
train_controlnet_sdxl_orig.py
--pretrained_model_name_or_path="model/stable-diffusion-xl-base-1.0"
--pretrained_vae_model_name_or_path="model/sdxl-vae-fp16-fix"
--output_dir="output"
--train_data_dir="fillsmall2"
--cache_dir="cachedir"
--mixed_precision="fp16"
--resolution=512
--learning_rate=1e-5
--max_train_steps=10000
--train_batch_size=1
--gradient_accumulation_steps=1
--max_grad_norm=1.0
--seed=42
--checkpointing_steps=1000
--enable_xformers_memory_efficient_attention
--use_8bit_adam
--report_to="tensorboard"
--lr_scheduler="constant_with_warmup"
--lr_warmup_steps=100
--logging_dir="logs"
--validation_image
"validation_images/1.png"
"validation_images/2.png"
"validation_images/3.png"
--validation_prompt
"red circle with blue background"
"green circle with brown background"
"blue circle with white background"
--validation_steps=10
--num_validation_images=2
```
Open http://localhost:6006/ (Tensorboard)

### Logs

```shell

```

### System Info

Diffusers version: 0.34.0.dev0
Torch version: 2.7.0+cu126
XFormers version: 0.0.30
Python version: 3.10.11
Windows 10

### Who can help?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.