huggingface / huggingface/diffusers
How about forcing the first and last block on device when groupoffloading is used?
Offen
#11,966
16 Kommentare
0 Reaktionen
1 zugewiesene Person
Beansprucht von @a-r-r-o-w
Auf GitHub ansehen
contributions-welcome
group-offloading
stale
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
**Is your feature request related to a problem? Please describe.**
When group offloading is enabled, the offload and onload cannot be streamed between steps and this is really a big time comsuming problem.
**Describe the solution you'd like.**
Is it possible to add an option that could make the first and last block forced on device to avoid offload and onload?
@a-r-r-o-w Could you please give some help? Thanks so much.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.