lllyasviel / lllyasviel/stable-diffusion-webui-forge
Interrogate Clip error - Expected all tensors to be on the same device
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
I am getting the following error when trying to use Interrogate.
```
[Unload] Trying to free 1024.00 MB for cuda:0 with 2 models keep loaded ... Current free memory is 2864.60 MB ... Done.
*** Error interrogating
Traceback (most recent call last):
File "D:\webui_forge_cu121_torch231\webui\modules\interrogate.py", line 191, in interrogate
caption = self.generate_caption(pil_image)
File "D:\webui_forge_cu121_torch231\webui\modules\interrogate.py", line 181, in generate_caption
caption = self.blip_model.generate(gpu_image, sample=False, num_beams=int(shared.opts.interrogate_clip_num_beams), min_length=int(shared.opts.interrogate_clip_min_length), max_length=shared.opts.interrogate_clip_max_length)
File "D:\webui_forge_cu121_torch231\webui\repositories\BLIP\models\blip.py", line 129, in generate
image_embeds = self.visual_encoder(image)
File "D:\webui_forge_cu121_torch231\system\python\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\webui_forge_cu121_torch231\system\python\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "D:\webui_forge_cu121_torch231\webui\repositories\BLIP\models\vit.py", line 185, in forward
x = torch.cat((cls_tokens, x), dim=1)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with modules/interrogate.py at interrogate and generate_caption, then follow the call into repositories/BLIP/models/blip.py and vit.py. Reproduce Interrogate with the reported setup and inspect where the image and model tensors are placed. Done means the caption generation no longer fails with a CPU/cuda:0 device mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100