huggingface / huggingface/diffusers
[modular] Migrate ComponentsManager offload hooks to ModelHook / HookRegistry
Offen
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
follow up TODO on https://github.com/huggingface/diffusers/pull/14304/
the auto-offload hooks used in components mannager are currently accelerate-style hooks: they implement the OOM retry by manually wrapping module entry points, since accelerate's pre_forward/post_forward can't see an exception raised by the forward pass itself.
diffusers now has its own hook system with a new_forward around-hook that covers exactly this case. We should look into migrate the offload hooks to it
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.