huggingface / huggingface/diffusers
Support out_dim argument for Attention block
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
**Is your feature request related to a problem? Please describe.**
When i feed the `out_dim` argument in `__init__` in [Attention block](https://github.com/huggingface/diffusers/blob/b69fd990ad8026f21893499ab396d969b62bb8cc/src/diffusers/models/attention_processor.py#L114) it will raise the shape error, because the `query_dim != out_dim`. In this case, the following code try to keep the given channel of `hidden_states`.
> https://github.com/huggingface/diffusers/blob/b69fd990ad8026f21893499ab396d969b62bb8cc/src/diffusers/models/attention_processor.py#L1393
But it should change the channel as the output of `hidden_states = attn.to_out[0](hidden_states)`.
**Describe the solution you'd like.**
I suggest the change of code base : https://github.com/huggingface/diffusers/blob/b69fd990ad8026f21893499ab396d969b62bb8cc/src/diffusers/models/attention_processor.py#L1393
to `hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, -1, height, width)`, then it will respect the channel of `hidden_states`.
Maybe I will make a PR later.
**Describe alternatives you've considered.**
None.
**Additional context.**
None.
Beitragsleitfaden
Rechercherichtung
Start in src/diffusers/models/attention_processor.py at the Attention block and the referenced processing code around line 1393. Reproduce the shape error with query_dim != out_dim, then verify that the attention output preserves the requested channel dimension and that existing attention behavior remains unchanged.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning
- Issue-Typ
- Feature
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Klar beschrieben
- Anfängerfreundlichkeit
- 48/100