huggingface / huggingface/diffusers

Attention masks are missing in SD3 to mask out text padding tokens

Open
#8,673 9 comments 0 reactions 0 assignees View on GitHub
contributions-welcome wip
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

In the attention implementation of SD3, attention masks currently are not used. This will result in inconsistent outputs for the different values `max_seq_length` where padding exists in text tokens as the attention scores of padding tokens are non-zero. This issue has been discussed in https://github.com/huggingface/diffusers/discussions/8628, and is created to track the progress of fixing this problem.

Thanks @sayakpaul for the discussion.

### Reproduction

n/a

### Logs

_No response_

### System Info

n/a

### Who can help?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.