playground cuda error

https://github.com/huggingface/diffusers/blob/896fb6d8d7c10001eb2a92568be7b4bd3d5ddea3/src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_img2img.py#L728

When I initialize without using `to('cuda')`, the model exists in the cpu, so here self.device gets the handle on the cpu, refer to the code above, whether we change it to `device`

## error msg:
init_latents = (init_latents - latents_mean) * self.vae.config.scaling_factor / latents_std
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

## os:
diffusers==diffusers-0.28.2

## this is my code:

```
import torch
from PIL import Image
from diffusers import StableDiffusionXLImg2ImgPipeline, EDMDPMSolverMultistepScheduler

sd_xl_path = '../models/Stable-diffusion/playground-v2.5-xl-aesthetic'
img_path = './image_1.png'

pipe = StableDiffusionXLImg2ImgPipeline.from_pretrained(sd_xl_path,
torch_dtype=torch.float16,
add_watermarker=False,
)

pipe.enable_model_cpu_offload()
pipe.scheduler = EDMDPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)

image = Image.open(img_path)

images = pipe(prompt='a beautiful girl',
image=image,
guidance_scale=3.0,
num_inference_steps=30,
strength=0.5
).images
print(images)

```

Start at src/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl_img2img.py around line 728 and reproduce the provided StableDiffusionXLImg2ImgPipeline example with enable_model_cpu_offload(). Inspect the tensors used in the failing init_latents expression. Done means the example completes without a CPU/CUDA device mismatch on diffusers 0.28.2.

Rédigé par le modèle d'indexation à partir du texte de l'issue.