Support int8 inference for diffusion models
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Feature Idea
Dear authors, thank you for your significant contributions to make diffusion models more accessible and user-friendly. I currently have some concerns regarding the inference precision (dtype) for diffusion models.
The weight_dtype parameter in UNETLoader (nodes.py) currently only supports the following precision formats: ["default", "fp8_e4m3fn", "fp8_e4m3fn_fast", "fp8_e5m2"].
Regarding numerical precision, research (e.g., https://arxiv.org/pdf/2303.17951) indicates that FP8 quantization tends to produce inferior model performance in both accuracy and efficiency compared to INT8.
I have two questions:
1. Technical Feasibility: Would it be possible to implement INT8 inference support for diffusion models in ComfyUI?
2. Implementation Considerations: Are there specific technical or practical reasons why INT8 inference isn't currently supported?
I look forward to your response.
Best regards
### Existing Solutions
_No response_
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.