huggingface / huggingface/diffusers
[Tracker] use micro-conditioning for the SDXL trainers
- Langage dominant
- Python
- Étoiles
- 34.5k
- Forks
- 7.3k
- Merge moyen
- 3 j 3 h
- PR mergées (30 j)
- 91
Description
SDXL makes use of micro-conditioning, and it does have quite a bit of an effect on the end results. For more details, refer to the paper [here](https://arxiv.org/abs/2307.01952).
Currently, not all of our SDXL trainers don't make use of micro-conditioning. So, it'd be nice to have micro-conditioning support as in https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora_sdxl.py.
Below is a list of the training scripts where we'd like to have this change incorporated:
- [ ] [DreamBooth SDXL LoRA](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora_sdxl.py)
- [ ] [SDXL LCM-LoRA](https://github.com/huggingface/diffusers/blob/main/examples/consistency_distillation/train_lcm_distill_lora_sdxl.py) (it needs to be slightly refactored as done in https://github.com/huggingface/diffusers/pull/6547 by @haofanwang)
- [ ] [ControlNet SDXL](https://github.com/huggingface/diffusers/blob/main/examples/controlnet/train_controlnet_sdxl.py)
- [ ] [T2I Adapter SDXL](https://github.com/huggingface/diffusers/blob/main/examples/t2i_adapter/train_t2i_adapter_sdxl.py)
- [x] [Textual inversion SDXL](https://github.com/huggingface/diffusers/blob/main/examples/textual_inversion/textual_inversion_sdxl.py) (same as what's mentioned for SDXL LCM-LoRA above)
- [x] [Advanced SDXL trainer](https://github.com/huggingface/diffusers/blob/main/examples/advanced_diffusion_training/train_dreambooth_lora_sdxl_advanced.py)
Feel free to open PRs targeting only ONE example at a time and tag me. Please also share an example training command while submitting the PRs. The command doesn't have to run the training for a large number of steps. Anything in the range of [4, 10] should suffice.
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez par comparer les scripts non cochés examples/dreambooth/train_dreambooth_lora_sdxl.py, examples/consistency_distillation/train_lcm_distill_lora_sdxl.py, examples/controlnet/train_controlnet_sdxl.py et examples/t2i_adapter/train_t2i_adapter_sdxl.py avec examples/text_to_image/train_text_to_image_lora_sdxl.py. Implémentez micro-conditioning sur un seul exemple à la fois, puis soumettez une commande d’entraînement d’exemple utilisant 4–10 étapes et vérifiez que l’élément de la checklist sélectionné est couvert.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python, pytorch
- Domaine
- machine-learning
- Type d'issue
- Fonctionnalité
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- Active
- Clarté
- Plutôt claire
- Accessibilité débutants
- 65/100