huggingface / huggingface/diffusers
[Modular] support group offloading in auto offloading
Abierto
- Lenguaje dominante
- Python
- Estrellas
- 34.5k
- Forks
- 7.3k
- Merge medio
- 3 d 3 h
- PR fusionados (30 d)
- 91
Descripción
Follow-up to #14304.
`enable_auto_cpu_offload` recommends group offloading for a model that OOMs with everything else already offloaded ("it does not fit on the device by itself"). but currently we cannot use group offloading and auto offloading togehter, should add support once #14304 is in
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.