huggingface / huggingface/diffusers
[modular] Migrate ComponentsManager offload hooks to ModelHook / HookRegistry
Aperta
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
follow up TODO on https://github.com/huggingface/diffusers/pull/14304/
the auto-offload hooks used in components mannager are currently accelerate-style hooks: they implement the OOM retry by manually wrapping module entry points, since accelerate's pre_forward/post_forward can't see an exception raised by the forward pass itself.
diffusers now has its own hook system with a new_forward around-hook that covers exactly this case. We should look into migrate the offload hooks to it
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.