huggingface / huggingface/diffusers
`StateManager`, `Hook`, `State`: missing context setting (cond or uncond) in pipelines
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
In diffusion denoising model, we usually use `classifier free guidance` to control qualities. With [State Manager](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/hooks/hooks.py#L42), we want each context hold different `state`, so we have to set which context when calling `self.transformer`, here is [a good example](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage.py#L797)
But some popular pipelines (Flux2) or pipeline variants (Kontext, Image to image, Inpainting) don't implement this. For example, [Flux T2I](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux.py#L943) has it while [Flux Kontext](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux_kontext.py#L1088) is missing
A Fix should be simple but require us to check all pipelines
```python
with self.transformer.cache_context("cond"):
noise_pred = self.transformer(...)
with self.transformer.cache_context("uncond"):
neg_noise_pred = self.transformer(...)
```
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start with src/diffusers/hooks/hooks.py and compare the context handling in pipeline_hunyuanimage.py, pipeline_flux.py, and pipeline_flux_kontext.py. Then inspect the other Flux2 and pipeline variant entry points mentioned in the issue for conditional and unconditional transformer calls. Done means affected pipelines consistently set the intended context for both calls and their existing tests pass.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 38/100