[SDXL] Sample generation during training not working

Here are examples of samples that was generated when i trained a short test dreambooth for just 400 steps(sample every 100 steps)

But when i ran the the minimal sdxl inference script on the model after 400 steps i got

I think part of the problem is samples are generated at a fixed 512x512, sdxl did not generate that good images for 512x512 in general. But don't think that is the main problem as i tried just changing that in the sampling code and images are still messed up

Reproduce the 400-step SDXL DreamBooth training run and compare its training-time samples with the minimal SDXL inference script output. Start by reading the sampling code, including its fixed 512x512 settings, and test whether changing the sampling resolution affects the result. Done means identifying and correcting the cause of the malformed samples, with matching validation images from both paths.

Written by the indexing model from the issue text.