huggingface / huggingface/diffusers

`StateManager`, `Hook`, `State`: missing context setting (cond or uncond) in pipelines

Open
#12,760 4 comments 0 reactions 0 assignees View on GitHub
contributions-welcome stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

In diffusion denoising model, we usually use `classifier free guidance` to control qualities. With [State Manager](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/hooks/hooks.py#L42), we want each context hold different `state`, so we have to set which context when calling `self.transformer`, here is [a good example](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage.py#L797)

But some popular pipelines (Flux2) or pipeline variants (Kontext, Image to image, Inpainting) don't implement this. For example, [Flux T2I](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux.py#L943) has it while [Flux Kontext](https://github.com/huggingface/diffusers/blob/152f7ca357c066c4af3d1a58cdf17662ef5a2f87/src/diffusers/pipelines/flux/pipeline_flux_kontext.py#L1088) is missing

A Fix should be simple but require us to check all pipelines
```python
with self.transformer.cache_context("cond"):
noise_pred = self.transformer(...)
with self.transformer.cache_context("uncond"):
neg_noise_pred = self.transformer(...)
```

Contributor guide

Open the contributing guide

Research direction

Start with src/diffusers/hooks/hooks.py and compare the context handling in pipeline_hunyuanimage.py, pipeline_flux.py, and pipeline_flux_kontext.py. Then inspect the other Flux2 and pipeline variant entry points mentioned in the issue for conditional and unconditional transformer calls. Done means affected pipelines consistently set the intended context for both calls and their existing tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.