Project-MONAI / Project-MONAI/tutorials

Clarification on mutually exclusive use of self-attention and cross-attention in DiffusionModelUNetMaisi

Offen
#1,993 4 Kommentare 0 Reaktionen 3 zugewiesene Personen Auf GitHub ansehen

@dongyang0122 arbeitet bereits daran.

Seit 09.6.2025.

Vorherrschende Sprache
Jupyter Notebook
Sterne
2.5k
Forks
803
Ø Merge
6 T. 22 Std.
Gemergte PRs (30 T.)
3

Beschreibung

Hi MONAI team,

while reading through the implementation of DiffusionModelUNetMaisi, I noticed the following logic for enabling attention at each level:

with_attn = attention_levels[i] and not with_conditioning
with_cross_attn = attention_levels[i] and with_conditioning

This effectively means that self-attention is never used when the model is in conditioning mode (with_conditioning=True), even if attention_levels[i] is True.

Is this behavior intentional?

In other diffusion-based architectures such as Stable Diffusion, it is common practice to enable both self-attention and cross-attention simultaneously within the same layers.

Would it be acceptable, or even recommended, to modify the logic as follows to allow both mechanisms in parallel?
Or is there a specific reason this mutual exclusivity was enforced?

Looking forward to your insights, and thank you for the great work on this model!

Best regards,
Daniele Molino

Beitragsleitfaden

Beitragsleitfaden öffnen

Erste Schritte

  1. Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
  3. Forke das Repository und arbeite in einem Branch.
  4. Öffne einen Pull Request, der die Issue-Nummer nennt.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.