huggingface / huggingface/diffusers
PixArt Sigma PEFT LoRA loader support
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
**Is your feature request related to a problem? Please describe.**
Currently, the upstream PixArt trainer does this:
```py
transformer = get_peft_model(transformer, lora_config)
if args.mixed_precision == "fp16":
# only upcast trainable parameters (LoRA) into fp32
cast_training_params(transformer, dtype=torch.float32)
transformer.print_trainable_parameters()
# 10. Handle saving and loading of checkpoints
# `accelerate` 0.16.0 will have better support for customized saving
if version.parse(accelerate.__version__) >= version.parse("0.16.0"):
# create custom saving & loading hooks so that `accelerator.save_state(...)` serializes in a nice format
def save_model_hook(models, weights, output_dir):
if accelerator.is_main_process:
transformer_ = accelerator.unwrap_model(transformer)
lora_state_dict = get_peft_model_state_dict(transformer_, adapter_name="default")
StableDiffusionPipeline.save_lora_weights(os.path.join(output_dir, "transformer_lora"), lora_state_dict)
# save weights in peft format to be able to load them back
transformer_.save_pretrained(output_dir)
for _, model in enumerate(models):
# make sure to pop weight so that corresponding model is not saved again
weights.pop()
```
**Describe the solution you'd like.**
I would like for the PixArtSigmaPipeline to have the necessary mix-ins and inference support instead.
**Describe alternatives you've considered.**
I have considered avoiding the pipeline methods and using workarounds like upstream does, but instead I would prefer consistency for user experience.
Beitragsleitfaden
Rechercherichtung
Start by reading the PixArtSigmaPipeline and the upstream PixArt trainer flow shown in the issue, especially its PEFT LoRA setup and checkpoint hooks. Determine the pipeline mix-ins and inference behavior needed for loading PixArt Sigma LoRA weights, then verify that the pipeline provides a consistent alternative to the trainer workarounds.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning
- Issue-Typ
- Feature
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Ruhig
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 48/100