Add GGUF Workflow Templates
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Feature Idea
# Problem with FP8
Usually workflow use safetensor clip and model using mostly FP16 or BF16 types that use around 2B per parameter but is broadly accelerated by GPUs.
To reduce memory footprint and run bigger models it's common to see models quantized to some FP8/FP4 type, and works with usually no change to the workflows. But that type isn't supported on some (many) GPUs, so it's upcasted to BF16 or FP16 acceleration negating the memory footprint advantage.
Some GPUs like the 7900XTX with ROCm have an hard time accelerating FP8, but have competent INT8 acceleration, both in Vulkan to acclerate llama.cpp and get good performance on quantized LLMs, and with ROCm to acclerate GGUF quants of models.
# GGUF Workflows
The suggestion is to add templates using [GGUF loaders ](https://github.com/city96/ComfyUI-GGUF) and pointing to GGUF quantizatized models to download for some image models like Flux, Zimage and Qwen Edit. I believe this would be useful to user across all GPUs to see if they caan get better performance with FP or Q type models.
Below an example using GGUF for CLIP and Model for Zimage Turbo, that with my 7900XTX ROCm under windows achieves much better performance than the BF16 models.
### Existing Solutions
Workflow using safetensors FP32 BF16 or FP16
Contributor guide
Assessment
This issue has not been assessed yet.