huggingface / huggingface/diffusers
Is Lumina2Pipeline's mu calculation correct?
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
### Describe the bug
Description
While reviewing the current main-branch implementation of pipeline_lumina2, I noticed a potential bug in the calculation of mu within the pipeline's __call__.
In the following section of the code:
https://github.com/huggingface/diffusers/blob/5ffb65803d0ddc5e3298c35df638ceed5e580922/src/diffusers/pipelines/lumina2/pipeline_lumina2.py#L484-L503
The latent tensor appears to have the shape:
(batch_size, num_channels_latents, height, width)
However, later in the same file:
https://github.com/huggingface/diffusers/blob/5ffb65803d0ddc5e3298c35df638ceed5e580922/src/diffusers/pipelines/lumina2/pipeline_lumina2.py#L699-L706
the value latent.shape[1] (i.e., num_channels_latents) is passed as the argument for image_seq_len when computing mu.
This seems incorrect, since image_seq_len should represent the number of image tokens or sequence length, not the number of latent channels.
Expected Behavior
image_seq_len should likely correspond to the number of spatial tokens derived from (height, width) (or another tokenization step), rather than the number of latent channels.
Actual Behavior
The current implementation uses latent.shape[1] as image_seq_len, which likely leads to unintended behavior in the computation of mu and subsequent sampling steps.
Suggested Fix
Review the logic where image_seq_len is passed, and ensure it reflects the correct sequence length dimension (possibly derived from spatial resolution or token count, rather than channel count).
### Reproduction
At the moment, I don’t have a copy/paste runnable MRE because this was identified via manual logic review rather than reproducing the behavior in a runtime environment.
### Logs
```shell
```
### System Info
Diffusers==0.36.0
Python==3.13
### Who can help?
_No response_
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start in src/diffusers/pipelines/lumina2/pipeline_lumina2.py, reading the __call__ sections around lines 484-503 and 699-706. Trace the latent shape and the value supplied as image_seq_len, then verify that mu uses the intended image token or sequence length and add or update coverage for the corrected behavior.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python
- Ambito
- machine-learning
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100