Comfy-Org / Comfy-Org/ComfyUI

Black Image SDXL + Xinsir Controlnet

Open
#13,166 2 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

I'm not sure why, but after not using SDXL / Pony / Illust models for a few months, every time I generate an image now, it turns black.

I'm on the latest ComfyUI master and still using the same launch args. I'm pretty sure that a few months ago it worked fine. The only thing I've been doing is routinely updating ComfyUI.

```py
python main.py --use-sage-attention --fast fp16_accumulation fp8_matrix_mult --preview-method latent2rgb --disable-api-nodes
```

In the latent preview, I can see some preview during the first 1–2 steps, then it suddenly goes completely black.

And, when I remove --use-sage-attention flags, everything works fine.

Also, when I use the KJ Checkpoint Loader and enable Sage Attention from that node, it works as well.

So now I'm wondering, is there something wrong with Comfy implementation of --use-sage-attention?

I don’t really like relying on too many custom nodes. If there’s a native option, I prefer to use that. But in this case, it seems like if I want the benefits of Sage attn, I have to use the KJ Checkpoint Loader.

There's also some error before the prompt executed logs.

### Actual Behavior

it shouldn’t randomly produce black outputs

### Steps to Reproduce

Run SDXL workflow with
```py
python main.py --use-sage-attention --fast fp16_accumulation fp8_matrix_mult --preview-method latent2rgb --disable-api-nodes
```

### Debug Logs

```powershell
got prompt

0: 1024x800 1 Not very manly man face, 66.9ms
Speed: 31.7ms preprocess, 66.9ms inference, 25.8ms postprocess per image at shape (1, 3, 1024, 800)

0: 1024x800 1 Not very manly man face, 12.6ms
Speed: 5.0ms preprocess, 12.6ms inference, 2.5ms postprocess per image at shape (1, 3, 1024, 800)
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Using sage attention mode: sageattn_qk_int8_pv_fp16_triton
Requested to load SDXLClipModel
Model SDXLClipModel prepared for dynamic VRAM loading. 1560MB Staged. 264 patches attached.
Model SDXLClipModel prepared for dynamic VRAM loading. 1560MB Staged. 264 patches attached.
Requested to load AutoencoderKL
0 models unloaded.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
0 models unloaded.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
# 😺dzNodes: LayerStyle -> ImageRemoveAlpha Processed 1 image(s).
Requested to load SDXL
Requested to load ControlNet
Model SDXL prepared for dynamic VRAM loading. 4897MB Staged. 722 patches attached.
Model ControlNet prepared for dynamic VRAM loading. 2397MB Staged. 0 patches attached.
27%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 8/30 [00:08<00:21, 1.01it/s]Interrupting prompt 155d8b7d-c045-416a-b4bc-177b211fc703
27%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 8/30 [00:08<00:23, 1.06s/it]
Processing interrupted
Prompt executed in 16.90 seconds
got prompt
0 models unloaded.
Model SDXL prepared for dynamic VRAM loading. 4897MB Staged. 722 patches attached.
Model ControlNet prepared for dynamic VRAM loading. 2397MB Staged. 0 patches attached.
100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 30/30 [00:21<00:00, 1.40it/s]
0 models unloaded.
Model AutoencoderKL prepared for dynamic VRAM loading. 159MB Staged. 0 patches attached.
Thread 0 error: Array must not contain infs or NaNs
F:\AI\ComfyUI-Nightly-AlwaysUpToDate\ComfyUI\nodes.py:1662: RuntimeWarning: invalid value encountered in cast
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
Prompt executed in 23.28 seconds
```

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the SDXL workflow with the listed main.py arguments, then compare it with runs without --use-sage-attention and with Sage Attention enabled through the KJ Checkpoint Loader. Start with the logged NaN failure at nodes.py:1662 and the preceding Sage Attention output. Done means the native option no longer produces infs or NaNs and the generated image is not black.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.