Flux fill is prone to crashing

### Expected Behavior

Normal output.

### Actual Behavior

After a few drawings it occasionally looks like this：

### Steps to Reproduce

[flux fill test.json](https://github.com/user-attachments/files/17867007/flux.fill.test.json)

### Debug Logs

```powershell
got prompt
using extra model: /root/ComfyUI/models/sams/sam_hq_vit_l.pth

grounding-dino is using models/bert-base-uncased
final text_encoder_type: /root/ComfyUI/models/bert-base-uncased
# 😺dzNodes: LayerStyle -> SegmentAnythingUltra Processed 1 image(s).
loaded completely 18778.313576507568 18638.461921691895 True
torch.Size([1, 1, 1024, 768])
torch.Size([1, 1, 1024, 768])
100%|██████████| 20/20 [00:19<00:00, 1.03it/s]
Prompt executed in 48.17 seconds
```

### Other

_No response_