huggingface / huggingface/diffusers
Attention masks are missing in SD3 to mask out text padding tokens
- Langage dominant
- Python
- Étoiles
- 34.5k
- Forks
- 7.3k
- Merge moyen
- 3 j 3 h
- PR mergées (30 j)
- 91
Description
### Describe the bug
In the attention implementation of SD3, attention masks currently are not used. This will result in inconsistent outputs for the different values `max_seq_length` where padding exists in text tokens as the attention scores of padding tokens are non-zero. This issue has been discussed in https://github.com/huggingface/diffusers/discussions/8628, and is created to track the progress of fixing this problem.
Thanks @sayakpaul for the discussion.
### Reproduction
n/a
### Logs
_No response_
### System Info
n/a
### Who can help?
_No response_
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Start with the SD3 attention implementation and read the linked discussion for the existing context on padding tokens. Trace how attention scores are computed when text padding is present, then verify that outputs no longer vary with max_seq_length solely because of padding.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python, pytorch
- Domaine
- machine-learning
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 35/100