huggingface / huggingface/diffusers

`StateManager`, `Hook`, `State`: missing context setting (cond or uncond) in pipelines

Aperta
#12,760 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
contributions-welcome stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

In diffusion denoising model, we usually use `classifier free guidance` to control qualities. With [State Manager](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/hooks/hooks.py#L42), we want each context hold different `state`, so we have to set which context when calling `self.transformer`, here is [a good example](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage.py#L797)

But some popular pipelines (Flux2) or pipeline variants (Kontext, Image to image, Inpainting) don't implement this. For example, [Flux T2I](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux.py#L943) has it while [Flux Kontext](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux_kontext.py#L1088) is missing

A Fix should be simple but require us to check all pipelines
```python
with self.transformer.cache_context("cond"):
noise_pred = self.transformer(...)
with self.transformer.cache_context("uncond"):
neg_noise_pred = self.transformer(...)
```

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with src/diffusers/hooks/hooks.py and compare the context handling in pipeline_hunyuanimage.py, pipeline_flux.py, and pipeline_flux_kontext.py. Then inspect the other Flux2 and pipeline variant entry points mentioned in the issue for conditional and unconditional transformer calls. Done means affected pipelines consistently set the intended context for both calls and their existing tests pass.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.