huggingface / huggingface/diffusers

better reporting of errors when using TP

Abierto
#14,533 0 comentarios 0 reacciones 1 asignado Reclamado por @JingyaHuang Ver en GitHub
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

I think the following aren't implemented at the moment (which is fine; just flagging).

* **Sharded loading**. from_pretrained should stream shards straight to each rank's DTensor rather than materializing the full checkpoint then slicing — otherwise TP saves you nothing at load time. And check `save_pretrained` / state_dict calls `.full_tensor()` or uses DCP. I think we should at least raise when `save_pretrained()` is called in case TP is enabled?
* **LoRA loading**. for a colwise base layer, lora_A replicated + lora_B colwise; for rowwise, lora_A rowwise + lora_B replicated. If the plan doesn't cover PEFT layers, loading an adapter onto a TP model will either error or be wrong. I think we should detect if the model has `peft` layers injected and raise if TP is requested?
* **Quantization, offloading**. We should probably also raise when these are requested?

_Originally posted by @sayakpaul in https://github.com/huggingface/diffusers/pull/13718#discussion_r3662821938_

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.