Free model and node cache: Sudden RAM usages increase & freezing machine

### Expected Behavior

Clicking up `free model and node cache` should reduce the percentage of the RAM usages

### Actual Behavior

Reporting a behavior where the Ram was around 50%. I clicked on the `free model and node cache` and the ram shot up to 96% and froze.

Another issue:
free model and node cache” doesn't work on clicking once, may need to click multiple time

### Steps to Reproduce

> Run a workflow e.g. Flux workflow
> After generation, click on `free model and node cache`


### Debug Logs

```powershell
No errors or info about memory/model unload in the logs
```

### Other

OS: Ubuntu
OS Version: 22.04
Python version: 3.12.6 (main, Oct 25 2024, 13:38:40) [GCC 11.4.0]
CUDA: 12.4

Total VRAM 45373 MB, total RAM 31640 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA L40S : cudaMallocAsync
ComfyUI version: 0.3.12
ComfyUI-Manager (V2.36)

Command line args: `--listen --disable-smart-memory --disable-xformers --port 7860 `

cc: @Kanss-code