huggingface / huggingface/diffusers

[Quantization] Add support for Comfy Quants backend

Abierto
#14,705 3 comentarios 0 reacciones 1 asignado Reclamado por @PrakshaaleJain Ver en GitHub
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

Add support for the [Comfy Quant](https://github.com/Comfy-Org/comfy-quants) quantization toolkit

Supported formats: FP8 (E4M3/E5M2), INT8 (W8A8, tensorwise), MXFP8, NVFP4, INT4 (SVDQuant W4A4, AWQ W4A16).

comfy-quants is export-only. The inference needs to run with [`comfy-kitchen`](https://github.com/Comfy-Org/comfy-kitchen)

### Proposed approach

Add a `ComfyQuantConfig` / `ComfyQuantizer` that uses `comfy-kitchen` to wrap weights as a `QuantizedTensor` with the appropriate layout.

```python
from diffusers import FluxTransformer2DModel, ComfyQuantConfig

config = ComfyQuantConfig(compute_dtype=torch.bfloat16)
model = FluxTransformer2DModel.from_single_file(
"path/to/comfy_quant_checkpoint.safetensors",
quantization_config=config,
)
```

### Related
• comfy-quants https://github.com/Comfy-Org/comfy-quants
• comfy-kitchen https://github.com/Comfy-Org/comfy-kitchen

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.