huggingface / huggingface/diffusers
PixArt Sigma PEFT LoRA loader support
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
**Is your feature request related to a problem? Please describe.**
Currently, the upstream PixArt trainer does this:
```py
transformer = get_peft_model(transformer, lora_config)
if args.mixed_precision == "fp16":
# only upcast trainable parameters (LoRA) into fp32
cast_training_params(transformer, dtype=torch.float32)
transformer.print_trainable_parameters()
# 10. Handle saving and loading of checkpoints
# `accelerate` 0.16.0 will have better support for customized saving
if version.parse(accelerate.__version__) >= version.parse("0.16.0"):
# create custom saving & loading hooks so that `accelerator.save_state(...)` serializes in a nice format
def save_model_hook(models, weights, output_dir):
if accelerator.is_main_process:
transformer_ = accelerator.unwrap_model(transformer)
lora_state_dict = get_peft_model_state_dict(transformer_, adapter_name="default")
StableDiffusionPipeline.save_lora_weights(os.path.join(output_dir, "transformer_lora"), lora_state_dict)
# save weights in peft format to be able to load them back
transformer_.save_pretrained(output_dir)
for _, model in enumerate(models):
# make sure to pop weight so that corresponding model is not saved again
weights.pop()
```
**Describe the solution you'd like.**
I would like for the PixArtSigmaPipeline to have the necessary mix-ins and inference support instead.
**Describe alternatives you've considered.**
I have considered avoiding the pipeline methods and using workarounds like upstream does, but instead I would prefer consistency for user experience.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reading the PixArtSigmaPipeline and the upstream PixArt trainer flow shown in the issue, especially its PEFT LoRA setup and checkpoint hooks. Determine the pipeline mix-ins and inference behavior needed for loading PixArt Sigma LoRA weights, then verify that the pipeline provides a consistent alternative to the trainer workarounds.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Tranquilla
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 48/100