huggingface / huggingface/diffusers
[modular] Migrate ComponentsManager offload hooks to ModelHook / HookRegistry
Open
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
follow up TODO on https://github.com/huggingface/diffusers/pull/14304/
the auto-offload hooks used in components mannager are currently accelerate-style hooks: they implement the OOM retry by manually wrapping module entry points, since accelerate's pre_forward/post_forward can't see an exception raised by the forward pass itself.
diffusers now has its own hook system with a new_forward around-hook that covers exactly this case. We should look into migrate the offload hooks to it
Contributor guide
Assessment
This issue has not been assessed yet.