Comfy-Org / Comfy-Org/ComfyUI

Support int8 inference for diffusion models

Open
#9,070 4 comments 3 reactions 0 assignees View on GitHub
bug-cop:triaged Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Feature Idea

Dear authors, thank you for your significant contributions to make diffusion models more accessible and user-friendly. I currently have some concerns regarding the inference precision (dtype) for diffusion models.

The weight_dtype parameter in UNETLoader (nodes.py) currently only supports the following precision formats: ["default", "fp8_e4m3fn", "fp8_e4m3fn_fast", "fp8_e5m2"].

Regarding numerical precision, research (e.g., https://arxiv.org/pdf/2303.17951) indicates that FP8 quantization tends to produce inferior model performance in both accuracy and efficiency compared to INT8.

I have two questions:
​​1. Technical Feasibility​​: Would it be possible to implement INT8 inference support for diffusion models in ComfyUI?
2. ​​Implementation Considerations​​: Are there specific technical or practical reasons why INT8 inference isn't currently supported?

I look forward to your response.
Best regards

### Existing Solutions

_No response_

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.