[Bug]: controlnet -photomaker didn't work （attached screenshot）

Nobody has claimed this yet.

### Checklist

- [X] The issue exists after disabling all extensions
- [ ] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

controlnet -photomaker didn't work

### Steps to reproduce the problem

1. Put the photomaker model into the model file of controlnet
2. Upload multiple pictures to photomaker
3. Check the keywords required for photomaker to start

### What should have happened?

Generate a graphic with the faces of the people I uploaded

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2024-02-06-16-36.json](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14183524/sysinfo-2024-02-06-16-36.json)

### Console logs

```Shell
2024-02-07 00:30:30,068 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
To load target model BaseModel
Begin to load 1 model
unload clone 2
Moving model(s) has taken 1.02 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 13.78it/s]
Token merging is under construction now and the setting will not take effect.██████████| 20/20 [00:01<00:00, 15.72it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.34it/s]
Token merging is under construction now and the setting will not take effect.██████████| 20/20 [00:01<00:00, 15.72it/s]
2024-02-07 00:32:28,303 - ControlNet - INFO - ControlNet Input Mode: InputMode.MERGE
2024-02-07 00:32:28,304 - ControlNet - INFO - Try to read image: C:\Users\FUNGAM~1\AppData\Local\Temp\gradio\bf84b36caa276a7e5b5926d32843b95aca4d5469\1570732594_instagram-jelodi-jung-68.jpg
2024-02-07 00:32:28,348 - ControlNet - INFO - Try to read image: C:\Users\FUNGAM~1\AppData\Local\Temp\gradio\a0f23073960158943590e0b4d1071243311af046\9b69603b5c64b45c9d16d7fc0ff6e75e--elodie-yung-human-body.jpg_http___avatars.mds.yandex.jpg
2024-02-07 00:32:28,360 - ControlNet - INFO - Try to read image: C:\Users\FUNGAM~1\AppData\Local\Temp\gradio\f8aa0e1153a28013262618a7163ec5566778559c\1_16a083549e9.1547137_3644485459_16a083549e9_large.jpg
2024-02-07 00:32:28,374 - ControlNet - INFO - Try to read image: C:\Users\FUNGAM~1\AppData\Local\Temp\gradio\9a53097bb068ae6aefb15890488f1e02aa2cedeb\1570732590_instagram-jelodi-jung-19.jpg
2024-02-07 00:32:28,484 - ControlNet - INFO - Using preprocessor: ClipVision (Photomaker)
2024-02-07 00:32:28,484 - ControlNet - INFO - preprocessor resolution = 0.5
2024-02-07 00:32:28,484 - ControlNet - INFO - Using preprocessor: ClipVision (Photomaker)
2024-02-07 00:32:28,484 - ControlNet - INFO - preprocessor resolution = 0.5
2024-02-07 00:32:28,484 - ControlNet - INFO - Using preprocessor: ClipVision (Photomaker)
2024-02-07 00:32:28,484 - ControlNet - INFO - preprocessor resolution = 0.5
2024-02-07 00:32:28,484 - ControlNet - INFO - Using preprocessor: ClipVision (Photomaker)
2024-02-07 00:32:28,484 - ControlNet - INFO - preprocessor resolution = 0.5
2024-02-07 00:32:29,241 - ControlNet - INFO - Current ControlNet PhotomakerPatcher: H:\webui_forge_cu121_torch21\webui\models\ControlNet\photomaker-v1.bin
To load target model SD1ClipModel
Begin to load 1 model
unload clone 2
Moving model(s) has taken 0.39 seconds
2024-02-07 00:32:29,685 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
To load target model BaseModel
Begin to load 1 model
unload clone 2
Moving model(s) has taken 1.09 seconds
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.91it/s]
Token merging is under construction now and the setting will not take effect.██████████| 20/20 [00:01<00:00, 14.34it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.01it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20
```

### Additional information

_No response_

Start by reproducing the issue through ControlNet's ClipVision (Photomaker) entry point, using the photomaker-v1.bin model and the multiple-image steps described. Review the console output and attached sysinfo to determine why generation completes without the uploaded faces; done means a graphic is generated using those faces.

Written by the indexing model from the issue text.