huggingface / huggingface/diffusers

Full support for Flux attention masking

Aperta
#10,194 2 commenti 4 reazioni 0 assegnatari Vedi su GitHub
stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

**Is your feature request related to a problem? Please describe.**
SimpleTuner/Kohya allow T5 attention masked training, however this is not currently supported natively in diffusers

**Describe the solution you'd like.**
Already implemented and used in Simpletuner and Kohya: https://github.com/bghira/SimpleTuner/blob/main/helpers/models/flux/transformer.py

**Describe alternatives you've considered.**
Recent implementation doesn't really solve the use case of using existing fine tunes with attention masking with diffusers
https://github.com/huggingface/diffusers/pull/10122

**Additional context.**
@yiyixuxu @bghira @AmericanPresidentJimmyCarter

@bghira's suggestion: "i'd suggested they add encoder_attention_mask and image_attention_mask and if image_attention_mask is None that they could then 1-fill those positions and just cat them together"

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by comparing the linked SimpleTuner implementation in helpers/models/flux/transformer.py with diffusers pull request 10122, then trace how Flux handles T5 attention masks. Done means diffusers natively supports attention-masked training for existing fine-tunes, including the proposed encoder_attention_mask and image_attention_mask behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.