huggingface / huggingface/diffusers

[Quantization] Add support for Comfy Quants backend

Ouverte
#14,705 3 commentaires 0 réactions 1 personne assignée Réclamée par @PrakshaaleJain Voir sur GitHub
Langage dominant
Python
Étoiles
34.5k
Forks
7.3k
Merge moyen
3 j 3 h
PR mergées (30 j)
91

Description

Add support for the [Comfy Quant](https://github.com/Comfy-Org/comfy-quants) quantization toolkit

Supported formats: FP8 (E4M3/E5M2), INT8 (W8A8, tensorwise), MXFP8, NVFP4, INT4 (SVDQuant W4A4, AWQ W4A16).

comfy-quants is export-only. The inference needs to run with [`comfy-kitchen`](https://github.com/Comfy-Org/comfy-kitchen)

### Proposed approach

Add a `ComfyQuantConfig` / `ComfyQuantizer` that uses `comfy-kitchen` to wrap weights as a `QuantizedTensor` with the appropriate layout.

```python
from diffusers import FluxTransformer2DModel, ComfyQuantConfig

config = ComfyQuantConfig(compute_dtype=torch.bfloat16)
model = FluxTransformer2DModel.from_single_file(
"path/to/comfy_quant_checkpoint.safetensors",
quantization_config=config,
)
```

### Related
• comfy-quants https://github.com/Comfy-Org/comfy-quants
• comfy-kitchen https://github.com/Comfy-Org/comfy-kitchen

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.