huggingface / huggingface/diffusers
Suggestion to add TEs
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
### Did you like the remote VAE solution?
I do... it's helpful when training/finetuning models, makes the infrastructure easier to set up.
### What can be improved about the current solution?
Adding TEs would be very helpful for a complete solution for auxiliary model off-loading during training.
Consider caching, then it could further simplify model fine-tuning setups.
### What other VAEs you would like to see if the pilot goes well?
StepVideo VAE: https://huggingface.co/stepfun-ai/stepvideo-t2v/tree/main/vae
HunYuan CLIP (for Stepvideo): https://huggingface.co/stepfun-ai/stepvideo-t2v/tree/main/hunyuan_clip
TE for LuminaVision2: [Gemma-2-2B](https://huggingface.co/google/gemma-2-2b)
### Notify the members of the team
@hlky @sayakpaul
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
The issue does not name files or tests; begin by locating the existing remote VAE solution and its model off-loading entry points. Clarify the scope for adding the listed text encoders and caching, then identify the relevant model-loading tests; done should cover the agreed auxiliary models and caching behavior.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- ai, machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100