Comfy-Org / Comfy-Org/ComfyUI

Black image (NaN) with int8_convrot diffusion models on gfx1201 / ROCm, but int8_convrot text encoder works fine

Open
#15,084 2 comments 0 reactions 0 assignees View on GitHub
User Support
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Your question

## Environment
- GPU: AMD Radeon RX 9070 XT (gfx1201, RDNA4)
- OS: Windows 11
- PyTorch: 2.9.1+rocm7.2.1, ROCm 7.2
- ComfyUI: v0.28.0-42-gf966a2b3
- comfy-kitchen: 0.2.22
- Python 3.12.10 (portable AMD build)
- Args: --windows-standalone-build --enable-triton-backend

## Description
Loading a Krea2 Turbo `int8_convrot` model in UNETLoader produces an all-black
image. Sampling completes without error; the failure surfaces at save time:

nodes.py:1682: RuntimeWarning: invalid value encountered in cast
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))

which indicates NaN/inf in the decoded tensor.

Notably, the *same* quantization format works correctly for the text encoder.
Only the diffusion model path is affected.

## Reproduction matrix (1024x1024, 16 steps, denoise 0.55, er_sde/simple)

| UNET | Text encoder | Result | s/it |
|-----------------|-----------------|--------|-------|
| int8_convrot | int8_convrot | black | 14.8 |
| int8_convrot | fp8_scaled | black | 16.6 |
| fp8_scaled | int8_convrot | OK | 36.7 |
| fp8_scaled | fp8_scaled | OK | 33.6 |

Models: Comfy-Org/Krea-2 (krea2_turbo_*, qwen3vl_4b_*), qwen_image_vae.

## Backend info
Native ops: convrot_w4a4, int8_tensorwise, float8_e4m3fn, float8_e5m2
emulated ops: nvfp4, mxfp8

comfy_kitchen `cuda` backend reports convrot capabilities but is disabled on
ROCm; only the `eager` backend is active for the convrot path.

## Note
int8_convrot is ~2.2x faster than fp8_scaled here, so this path is worth
fixing for RDNA4. Possibly related to #14864 (black image on Turing with
int4 models).

### Logs

```powershell

```

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the 1024x1024 matrix with the listed AMD, ROCm, PyTorch, and ComfyUI versions. Start at UNETLoader and the int8_convrot path in comfy-kitchen's eager backend, then inspect the invalid cast reported at nodes.py:1682. Done means int8_convrot diffusion sampling produces finite decoded values and a non-black image while the text-encoder path remains functional.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.