huggingface / huggingface/diffusers

PNDMScheduler Does Not Compatible with DDPMPipeline

Aperta
#7,354 9 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Describe the bug

I was trying to test different schedulers under DDPMPipeline. And an error occurred if I use PNDMScheduler beforehand I have found that PNDMScheduler should be compatible with DDPMPipeline following the official tutorial.
```Python
pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
pipeline.scheduler.compatibles
```
And the output is this:
```
[diffusers.schedulers.scheduling_ddim.DDIMScheduler,
diffusers.schedulers.scheduling_euler_discrete.EulerDiscreteScheduler,
diffusers.schedulers.scheduling_dpmsolver_multistep.DPMSolverMultistepScheduler,
diffusers.schedulers.scheduling_lms_discrete.LMSDiscreteScheduler,
diffusers.schedulers.scheduling_dpmsolver_singlestep.DPMSolverSinglestepScheduler,
diffusers.schedulers.scheduling_deis_multistep.DEISMultistepScheduler,
diffusers.utils.dummy_torch_and_torchsde_objects.DPMSolverSDEScheduler,
diffusers.schedulers.scheduling_k_dpm_2_discrete.KDPM2DiscreteScheduler,
diffusers.schedulers.scheduling_k_dpm_2_ancestral_discrete.KDPM2AncestralDiscreteScheduler,
diffusers.schedulers.scheduling_unipc_multistep.UniPCMultistepScheduler,
diffusers.schedulers.scheduling_heun_discrete.HeunDiscreteScheduler,
diffusers.schedulers.scheduling_euler_ancestral_discrete.EulerAncestralDiscreteScheduler,
diffusers.schedulers.scheduling_pndm.PNDMScheduler,
diffusers.schedulers.scheduling_ddpm.DDPMScheduler]
```
And the main error massage is that
```Python
File [c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\diffusers\pipelines\ddpm\pipeline_ddpm.py:117](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:117), in DDPMPipeline.__call__(self, batch_size, generator, num_inference_steps, output_type, return_dict)
[114](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:114) model_output = self.unet(image, t).sample
[116](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:116) # 2. compute previous image: x_t -> x_t-1
--> [117](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:117) image = self.scheduler.step(model_output, t, image, generator=generator).prev_sample
[119](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:119) image = (image / 2 + 0.5).clamp(0, 1)
[120](file:///C:/Users/User/miniconda3/envs/Pytorch/lib/site-packages/diffusers/pipelines/ddpm/pipeline_ddpm.py:120) image = image.cpu().permute(0, 2, 3, 1).numpy()
TypeError: PNDMScheduler.step() got an unexpected keyword argument 'generator'
```
In DDPMPipeline.scheduler.step( ) function, it takes `generator` as the argument. But in class diffusers.PNDMScheduler step( ) function shown in [official doc](https://huggingface.co/docs/diffusers/api/schedulers/pndm#diffusers.PNDMScheduler.step), the function only takes model_output (torch.FloatTensor), timestep (int), sample (torch.FloatTensor), return_dict (bool).
Moreover, I also find out that [HeunDiscreteScheduler](https://huggingface.co/docs/diffusers/api/schedulers/heun#diffusers.HeunDiscreteScheduler.step) has the same problem

### Reproduction

```python
from diffusers import DDIMScheduler
from diffusers import UNet2DModel
from diffusers import PNDMScheduler
model = UNet2DModel(
sample_size=config.image_size,
in_channels=1,
out_channels=1,
layers_per_block=2,
block_out_channels=(128,128,256,512),
down_block_types=(
"DownBlock2D",
"DownBlock2D",
"AttnDownBlock2D",
"DownBlock2D",
),
up_block_types=(
"UpBlock2D",
"AttnUpBlock2D",
"UpBlock2D",
"UpBlock2D",
),
)
noise_scheduler = PNDMScheduler()
pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
print(pipeline.scheduler.compatibles)
images = pipeline(
batch_size=1,
num_inference_steps = 50).images
```

### Logs

```shell
TypeError Traceback (most recent call last)
Input In [58], in ()
24 pipeline = DDPMPipeline(unet=model, scheduler=noise_scheduler)
25 pipeline.scheduler.compatibles
---> 26 images = pipeline(
27 batch_size=1,
28 num_inference_steps = 50).images

File c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\torch\utils\_contextlib.py:115, in context_decorator..decorate_context(*args, **kwargs)
112 @functools.wraps(func)
113 def decorate_context(*args, **kwargs):
114 with ctx_factory():
--> 115 return func(*args, **kwargs)

File c:\Users\User\miniconda3\envs\Pytorch\lib\site-packages\diffusers\pipelines\ddpm\pipeline_ddpm.py:117, in DDPMPipeline.__call__(self, batch_size, generator, num_inference_steps, output_type, return_dict)
114 model_output = self.unet(image, t).sample
116 # 2. compute previous image: x_t -> x_t-1
--> 117 image = self.scheduler.step(model_output, t, image, generator=generator).prev_sample
119 image = (image / 2 + 0.5).clamp(0, 1)
120 image = image.cpu().permute(0, 2, 3, 1).numpy()

TypeError: PNDMScheduler.step() got an unexpected keyword argument 'generator'
```

### System Info

I cannot use this command but I can show some info.
diffusers 0.26.3 pyhd8ed1ab_0 conda-forge
pytorch 2.0.1 py3.10_cuda11.8_cudnn8_0 pytorch
OS: Windows 11

### Who can help?

@yiyixuxu @Dn

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia in pipelines/ddpm/pipeline_ddpm.py, alla chiamata a scheduler.step, quindi esamina le implementazioni di step per PNDMScheduler e HeunDiscreteScheduler descritte nell’issue. Esegui nuovamente la riproduzione DDPMPipeline fornita e conferma che l’uso di questi scheduler non genera più un errore imprevisto relativo all’argomento con parola chiave generator.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
48/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.