huggingface / huggingface/diffusers

[modular] Migrate ComponentsManager offload hooks to ModelHook / HookRegistry

Aperta
#14,328 0 commenti 0 reazioni 1 assegnatario Rivendicata da @yiyixuxu Vedi su GitHub
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

follow up TODO on https://github.com/huggingface/diffusers/pull/14304/

the auto-offload hooks used in components mannager are currently accelerate-style hooks: they implement the OOM retry by manually wrapping module entry points, since accelerate's pre_forward/post_forward can't see an exception raised by the forward pass itself.

diffusers now has its own hook system with a new_forward around-hook that covers exactly this case. We should look into migrate the offload hooks to it

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.