Comfyui during ksampler is running too slow suddenly ( need 20 minutes) for flux gguf model

### Your question

I have install pytorch + cuda12.4 version, but in the nvidia-smi is always showing cuda 12.7

when I run the flux gguf workflow for Image to image
it show
> VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
> SELECTED: input2
> CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16

workflow I use：
[flux-gguf-q8-12gb-1.json](https://github.com/user-attachments/files/18458600/flux-gguf-q8-12gb-1.json)


I do not what to do to let it use GPU , can anybody help?

### Logs

```powershell
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
SELECTED: input2
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
clip missing: ['text_projection.weight']
Requested to load FluxClipModel_
loaded completely 9.5367431640625e+25 4777.53759765625 True
C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-GGUF\loader.py:65: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_numpy.cpp:212.)
torch_tensor = torch.from_numpy(tensor.data) # mmap

ggml_sd_loader:
0 471
8 304
1 5
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX
Requested to load AutoencodingEngine
0 models unloaded.
loaded completely 9.5367431640625e+25 159.87335777282715 True
SELECTED: input2
Requested to load Flux
loaded partially 1409.7749671936035 1409.744140625 0
Attempting to release mmap (293)
5%|████▏ | 1/20 [01:04<20:19, 64.20s/it]
```

### Other

_No response_