huggingface / huggingface/diffusers
[Modular] support group offloading in auto offloading
Open
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
Follow-up to #14304.
`enable_auto_cpu_offload` recommends group offloading for a model that OOMs with everything else already offloaded ("it does not fit on the device by itself"). but currently we cannot use group offloading and auto offloading togehter, should add support once #14304 is in
Contributor guide
Assessment
This issue has not been assessed yet.