huggingface / huggingface/diffusers
Full support for Flux attention masking
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
**Is your feature request related to a problem? Please describe.**
SimpleTuner/Kohya allow T5 attention masked training, however this is not currently supported natively in diffusers
**Describe the solution you'd like.**
Already implemented and used in Simpletuner and Kohya: https://github.com/bghira/SimpleTuner/blob/main/helpers/models/flux/transformer.py
**Describe alternatives you've considered.**
Recent implementation doesn't really solve the use case of using existing fine tunes with attention masking with diffusers
https://github.com/huggingface/diffusers/pull/10122
**Additional context.**
@yiyixuxu @bghira @AmericanPresidentJimmyCarter
@bghira's suggestion: "i'd suggested they add encoder_attention_mask and image_attention_mask and if image_attention_mask is None that they could then 1-fill those positions and just cat them together"
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by comparing the linked SimpleTuner implementation in helpers/models/flux/transformer.py with diffusers pull request 10122, then trace how Flux handles T5 attention masks. Done means diffusers natively supports attention-masked training for existing fine-tunes, including the proposed encoder_attention_mask and image_attention_mask behavior.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 35/100