huggingface / huggingface/controlnet_aux
Errors and Solutions Encountered When Using SamDetector.
- Dominant language
- Python
- Stars
- 497
- Forks
- 104
- PR merge metrics
- No merged PRs in 30d
Description
I encountered an error while using SamDetector, and the error message is as follows:
```
Traceback (most recent call last):
File "c:\PythonProgram\diffweb\test_seg.py", line 12, in <module>
processed = sam(image)
File "C:\PythonProgram\diffweb\lib\site-packages\controlnet_aux\segment_anything\__init__.py", line 76, in __call__
masks = self.mask_generator.generate(input_image)
// ... Omitted intermediate error information ...
File "C:\PythonProgram\diffweb\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py", line 274, in forward
(q @ k.transpose(-2, -1)) * self.scale
```
`RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!`
After some investigation, I found that this issue was caused by some parts of the model using GPU (cuda:0) for computation, while other parts were using the CPU. This led to a device mismatch problem when executing the attention mechanism.
To solve this issue, I added the following code before the line:
```
attn = (
(q @ k.transpose(-2, -1)) * self.scale
+
(self.attention_biases[:, self.attention_bias_idxs] if self.training else self.ab)
)
```
`self.ab = self.ab.to('cuda:0')
`By doing this, all tensors in the model are computed on the GPU, thereby avoiding the device mismatch problem.
I hope this solution will be helpful to others.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.