huggingface / huggingface/diffusers
Full support for Flux attention masking
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
**Is your feature request related to a problem? Please describe.**
SimpleTuner/Kohya allow T5 attention masked training, however this is not currently supported natively in diffusers
**Describe the solution you'd like.**
Already implemented and used in Simpletuner and Kohya: https://github.com/bghira/SimpleTuner/blob/main/helpers/models/flux/transformer.py
**Describe alternatives you've considered.**
Recent implementation doesn't really solve the use case of using existing fine tunes with attention masking with diffusers
https://github.com/huggingface/diffusers/pull/10122
**Additional context.**
@yiyixuxu @bghira @AmericanPresidentJimmyCarter
@bghira's suggestion: "i'd suggested they add encoder_attention_mask and image_attention_mask and if image_attention_mask is None that they could then 1-fill those positions and just cat them together"
Beitragsleitfaden
Rechercherichtung
Start by comparing the linked SimpleTuner implementation in helpers/models/flux/transformer.py with diffusers pull request 10122, then trace how Flux handles T5 attention masks. Done means diffusers natively supports attention-masked training for existing fine-tunes, including the proposed encoder_attention_mask and image_attention_mask behavior.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning
- Issue-Typ
- Feature
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 35/100