Image generation on 3090 is sometimes broken and worse than on 2060, and can't reproduce it

### Expected Behavior

I have a workflow that produces extremely different results on 2060 GPU on a different PC, and my 3090. This image looks correct, and was generated on 2060.


### Actual Behavior

This is what gets generated on my 3090, no matter what I do: updating pytorch, drivers, changing VAE, changing attention options, changing fp32/bf16 settings. This results in slight changes in the image but it remains broken. Btw, generating on CPU is completely broken.


### Steps to Reproduce

[ComfyUI_01254_.json](https://github.com/user-attachments/files/18609546/ComfyUI_01254_.json)

The model used is obsessionIllustrious_v31.safetensors, https://civitai.com/models/820208?modelVersionId=1136462

No custom nodes required

### Debug Logs

```powershell
@:~/github/ComfyUI$ python main.py --use-sage-attention --highvram --disable-all-custom-nodes
...
Checkpoint files will always be loaded safely.
Total VRAM 24135 MB, total RAM 64001 MB
pytorch version: 2.6.0+cu126
Set vram state to: HIGH_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
Using sage attention
ComfyUI version: 0.3.13
...
Skipping loading of custom nodes
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
loaded diffusion model directly to GPU
Requested to load SDXL
loaded completely 9.5367431640625e+25 4897.0483474731445 True
Requested to load SDXLClipModel
loaded completely 9.5367431640625e+25 1560.802734375 True
100%|█████████████████████████████████████████████████████████████████████████████| 25/25 [00:07<00:00, 3.40it/s]
Requested to load AutoencoderKL
loaded completely 9.5367431640625e+25 159.55708122253418 True
Prompt executed in 9.84 seconds
```

### Other

_No response_