huggingface / huggingface/diffusers

global, eager model weight GPU unloading

Aperta
#8,605 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub
stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

**What API design would you like to have changed or added to the library? Why?**

Most people expect `diffusers` and `transformers` "models" to be "unloaded" so that they can "just" "run" a "big" "pipeline" using their "VRAM" so that it "fits."

In other words, author a mixin that keeps track of *all* weights in Hugging Face hierarchy objects loaded onto the GPU; and when `forward` is called on *any* Hugging Face hierarchy object, moves weights in *other* objects being tracked to ordinary RAM. Essentially, this is sequential CPU offload for scopes larger than a Hugging Face hierarchy object.

**What use case would this enable or better enable? Can you give us a code example?**

The number of issues about GPU RAM usage scales linearly with adoption. You guys can't deal with the brain damage of having your Issues polluted by this.

Separately, it would eliminate the main source of toil for people who integrate `diffusers` into other products like ComfyUI.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by reviewing the proposed mixin design and how diffusers and transformers model hierarchies are used during forward calls. Define how tracked GPU weights would move to ordinary RAM across separate hierarchy objects, and validate that large pipelines can fit within available VRAM without disrupting existing integrations such as ComfyUI.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning, performance
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
20/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.