For models that take a long time to load

### Feature Idea

For models that take a long time to load, shouldn't they be retained separately when clearing GPU memory and RAM? For example, large language models loaded by DualCLIPLoader are very large and take a long time to load. Could an option be added at this node to specify the memory usage of the "remove model" setting? That's the gist of it. (Actually, I mainly want to be able to dynamically change this value at the code level so it doesn't fail to clear, but I don't want it to take a long time to load every time, especially with my limited 48GB of RAM.)

### Existing Solutions

_No response_

### Other

_No response_