Comfy-Org / Comfy-Org/ComfyUI

Add GGUF Workflow Templates

Open
#11,819 0 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Feature Idea

# Problem with FP8

Usually workflow use safetensor clip and model using mostly FP16 or BF16 types that use around 2B per parameter but is broadly accelerated by GPUs.

To reduce memory footprint and run bigger models it's common to see models quantized to some FP8/FP4 type, and works with usually no change to the workflows. But that type isn't supported on some (many) GPUs, so it's upcasted to BF16 or FP16 acceleration negating the memory footprint advantage.

Some GPUs like the 7900XTX with ROCm have an hard time accelerating FP8, but have competent INT8 acceleration, both in Vulkan to acclerate llama.cpp and get good performance on quantized LLMs, and with ROCm to acclerate GGUF quants of models.

# GGUF Workflows

The suggestion is to add templates using [GGUF loaders ](https://github.com/city96/ComfyUI-GGUF) and pointing to GGUF quantizatized models to download for some image models like Flux, Zimage and Qwen Edit. I believe this would be useful to user across all GPUs to see if they caan get better performance with FP or Q type models.

Below an example using GGUF for CLIP and Model for Zimage Turbo, that with my 7900XTX ROCm under windows achieves much better performance than the BF16 models.

Image

### Existing Solutions

Workflow using safetensors FP32 BF16 or FP16

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.