Comfy-Org / Comfy-Org/ComfyUI

Native INT8 ConvRot significantly slower than FP8 on A100 (Torch 2.10 + cu130)

Open
#14,824 23 comments 2 reactions 0 assignees View on GitHub
User Support
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Your question

### Problem
On A100 GPUs, the newly added native **INT8 ConvRot** models (introduced in v0.27.0) are **much slower** than FP8 (only 1/3 to 1/2 the speed of FP8), contrary to the claimed 1.5-2x+ speedup on most NVIDIA GPUs.

Tested models:
- Flux.2-Klein
- Qwen-Image-Edit (qwenedit)

**Environment:**
- GPU: NVIDIA A100 (80GB)
- PyTorch: 2.10 + cu130
- ComfyUI: v0.27.0)
- CUDA: 13.0

### Expected Behavior
According to the announcement and community benchmarks on RTX 20/30/40/50 series, INT8 ConvRot should provide:
- Better or comparable quality to FP8
- Significantly faster inference (especially on cards with strong INT8 Tensor Core support)

A100 has excellent INT8 Tensor Core performance, so it should benefit as well or even more.

### Actual Behavior
- INT8 ConvRot generation speed is only **33%~50%** of FP8.
- This matches reports from other users that A100 sees little to no speedup (or regression) with the current INT8 implementation.

### Additional Context
- ConvRot is a QuaRot variant using Hadamard rotation + row-wise INT8 to suppress outliers.
- The custom node https://github.com/BobJohnson24/ComfyUI-INT8-Fast/issues/71 also notes limited/no benefit on A100.
- The issue is likely due to kernel fusion, dispatch paths, or missing optimizations for Ampere datacenter GPUs (vs consumer RTX cards) in the current native implementation.

### Steps to Reproduce
1. Load an official INT8 ConvRot model (e.g. from Comfy-Org org on HF or modelscope).
2. Compare with the FP8 version of the same model using the same workflow.
3. Measure steps/second or total generation time.

### Suggested Fix / Help Wanted
- Improve kernel support / Triton kernels for A100/Ampere in native INT8 path.
- Better fallback or mixed-precision handling for datacenter GPUs.
- Profiling data or torch.compile integration improvements for ConvRot operations.

Would be great if the team could add A100-specific benchmarks or optimizations in future releases. Happy to provide more logs, profiler outputs, or test workflows.

### Logs

```powershell

```

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

No source file or test is named. Reproduce the comparison with the listed INT8 ConvRot and FP8 models on an A100 using the same workflow, then collect profiling data around the native INT8 path; done means identifying the cause and demonstrating improved INT8 performance against the FP8 baseline.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.