[Bug]: Unable to use locally stored BLIP models

Nobody has claimed this yet.

### Checklist

- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

I use the [clip-interrogator](https://github.com/pharmapsychotic/clip-interrogator-ext) extension in A1111 & by default the models are downloaded & stored at `C:\Users\.cache\huggingface\hub`, but Forge seems to download the models to `stable-diffusion-webui-forge\models\diffusers`. Please help me figure this out because these models are quite big & i can't afford redownloading all the models again. I'm guessing my solution would be to find a metadata file somewhere in Forge folder but i couldn't find anything.

### Steps to reproduce the problem

N/A

### What should have happened?

Forge should use the pre installed BLIP models.

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2024-07-17-04-48.json](https://github.com/user-attachments/files/16260346/sysinfo-2024-07-17-04-48.json)

### Console logs

```Shell
...
Loading CLIP Interrogator 0.6.0...
Loading caption model blip-base...
D:\AI\stable-diffusion-webui-forge\venv\lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
config.json: 100%|████████████████████████████████████████████████████████████████████████| 4.56k/4.56k [00:00

Start by tracing how the clip-interrogator extension loads the BLIP caption model and how huggingface_hub selects its cache, comparing C:\Users\.cache\huggingface\hub with models\diffusers. Done means Forge reuses the existing BLIP files instead of downloading them again; the issue provides no source file or test to run.

Written by the indexing model from the issue text.