huggingface / huggingface/diffusers

A bug about one-step inference in PixArtAlphaPipeline

Abierto
#8,689 6 comentarios 0 reacciones 0 asignados Ver en GitHub
bug stale
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

### Describe the bug

When implementing the PixArtAlphaPipeline, one step of inference was bound to the DMD, which is inappropriate. This resulted in errors in other one-step inference codes based on Pixar-alpha.

### Reproduction

`import torch
from diffusers import PixArtAlphaPipeline, LCMScheduler, Transformer2DModel

transformer = Transformer2DModel.from_pretrained(
"Luo-Yihong/yoso_pixart1024", torch_dtype=torch.float16).to('cuda')

pipe = PixArtAlphaPipeline.from_pretrained("PixArt-alpha/PixArt-XL-2-512x512",
transformer=transformer,
torch_dtype=torch.float16, use_safetensors=True)

pipe = pipe.to('cuda')
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.scheduler.config.prediction_type = "v_prediction"
generator = torch.manual_seed(318)
imgs = pipe(prompt="Pirate ship trapped in a cosmic maelstrom nebula, rendered in cosmic beach whirlpool engine, volumetric lighting, spectacular, ambient lights, light pollution, cinematic atmosphere, art nouveau style, illustration art artwork by SenseiJaye, intricate detail.",
num_inference_steps=1,
num_images_per_prompt = 1,
generator = generator,
guidance_scale=1.,
)[0]
imgs[0]
`
The code is not able to run for now.

### Logs

```shell
File D:\ComfyUI\venv\lib\site-packages\diffusers\pipelines\pixart_alpha\pipeline_pixart_alpha.py:942, in PixArtAlphaPipeline.call(self, prompt, negative_prompt, num_inference_steps, timesteps, sigmas, guidance_scale, num_images_per_prompt, height, width, eta, generator, latents, prompt_embeds, prompt_attention_mask, negative_prompt_embeds, negative_prompt_attention_mask, output_type, return_dict, callback, callback_steps, clean_caption, use_resolution_binning, max_sequence_length, **kwargs)
939 # compute previous image: x_t -> x_t-1
940 if num_inference_steps == 1:
941 # For DMD one step sampling: https://arxiv.org/abs/2311.18828
--> 942 latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs).pred_original_sample
943 else:
944 latents = self.scheduler.step(noise_pred, t, latents, **extra_step_kwargs, return_dict=False)[0]

AttributeError: 'LCMSchedulerOutput' object has no attribute 'pred_original_sample'
```

### System Info

The newest diffuser.

### Who can help?

@yiyixuxu @DN6

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Comienza en diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py alrededor de la línea 942 e inspecciona la rama de inferencia de un paso junto con la salida de LCMScheduler mostrada en el traceback. Ejecuta la reproducción proporcionada con num_inference_steps=1; se considera terminado cuando la pipeline ya no genera el error missing pred_original_sample para LCMScheduler.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python, pytorch
Área
machine-learning
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
43/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.