huggingface / huggingface/diffusers

How about forcing the first and last block on device when groupoffloading is used?

Aperta
#11,966 16 commenti 0 reazioni 1 assegnatario Rivendicata da @a-r-r-o-w Vedi su GitHub
contributions-welcome group-offloading stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

**Is your feature request related to a problem? Please describe.**
When group offloading is enabled, the offload and onload cannot be streamed between steps and this is really a big time comsuming problem.

**Describe the solution you'd like.**
Is it possible to add an option that could make the first and last block forced on device to avoid offload and onload?

@a-r-r-o-w Could you please give some help? Thanks so much.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.