huggingface / huggingface/diffusers
global, eager model weight GPU unloading
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
**What API design would you like to have changed or added to the library? Why?**
Most people expect `diffusers` and `transformers` "models" to be "unloaded" so that they can "just" "run" a "big" "pipeline" using their "VRAM" so that it "fits."
In other words, author a mixin that keeps track of *all* weights in Hugging Face hierarchy objects loaded onto the GPU; and when `forward` is called on *any* Hugging Face hierarchy object, moves weights in *other* objects being tracked to ordinary RAM. Essentially, this is sequential CPU offload for scopes larger than a Hugging Face hierarchy object.
**What use case would this enable or better enable? Can you give us a code example?**
The number of issues about GPU RAM usage scales linearly with adoption. You guys can't deal with the brain damage of having your Issues polluted by this.
Separately, it would eliminate the main source of toil for people who integrate `diffusers` into other products like ComfyUI.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reviewing the proposed mixin design and how diffusers and transformers model hierarchies are used during forward calls. Define how tracked GPU weights would move to ordinary RAM across separate hierarchy objects, and validate that large pipelines can fit within available VRAM without disrupting existing integrations such as ComfyUI.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning, performance
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 20/100