huggingface / huggingface/diffusers

Support for training with Grayscale images?

Open
#10,677 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

I am trying to train an unconditional diffusion model on grayscale images using your [pipeline](https://huggingface.co/docs/diffusers/training/unconditional_training). When running training with the default parameters I discovered inferred images that contained colour (specifically green). Where it learnt such colours from I do not know but I would predict the issue lies within the initial processing of the image set:

`images = [augmentations(image.convert("RGB")) for image in examples["image"]]`

as such I created a fork of this [repo ](https://github.com/DavidGill159/diffusers/tree/main/examples/unconditional_image_generation)and changed this line to:

`images = [augmentations(image.convert("L")) for image in examples["image"]]`

I also updated the model configuration (UNet2DModel) to work with single-channel inputs and outputs by setting `in_channels=1` and `out_channels=1` when initialising the model.

Am I on the right track? or does the resolution lie elsewhere? I also noticed the resolution of the inferred images is very poor; not on par with the training set. What parameters can I adjust to improve this?
**Ultimately I am interested in a diffusion model that focuses more on the textural composition of images, rather than the colou**r.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.