huggingface / huggingface/diffusers

[modular] Migrate ComponentsManager offload hooks to ModelHook / HookRegistry

Open
#14,328 0 comments 0 reactions 1 assignee Claimed by @yiyixuxu View on GitHub
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

follow up TODO on https://github.com/huggingface/diffusers/pull/14304/

the auto-offload hooks used in components mannager are currently accelerate-style hooks: they implement the OOM retry by manually wrapping module entry points, since accelerate's pre_forward/post_forward can't see an exception raised by the forward pass itself.

diffusers now has its own hook system with a new_forward around-hook that covers exactly this case. We should look into migrate the offload hooks to it

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.