huggingface / huggingface/diffusers
How about forcing the first and last block on device when groupoffloading is used?
Aperta
contributions-welcome
group-offloading
stale
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
**Is your feature request related to a problem? Please describe.**
When group offloading is enabled, the offload and onload cannot be streamed between steps and this is really a big time comsuming problem.
**Describe the solution you'd like.**
Is it possible to add an option that could make the first and last block forced on device to avoid offload and onload?
@a-r-r-o-w Could you please give some help? Thanks so much.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.