SAM only uses 1 gpu

Hi,
I tried to run SAM but I got an error about GPU memory. The problem is the SAM only used 1 GPU only. Before loading and running SAM, I did check the number of available gpus `torch.cuda.device_count: 2` but sill got the problem.

Plus I tried different methods such as `model = torch.nn.DataParallel(sam, [0,1])` and `model.to(device='cuda')`. But it didnt work.

`torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 10.96 GiB (GPU 0; 14.62 GiB total capacity; 2.56 GiB already allocated; 10.02 GiB free; 3.61 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
`

No repository file or test is identified. Start by reproducing the SAM run while checking torch.cuda.device_count and the reported DataParallel and model.to attempts; compare device placement and memory use on GPUs 0 and 1. Done means identifying whether the reported invocation can use both GPUs and resolving or clearly documenting the cause of the GPU 0 out-of-memory error.

Written by the indexing model from the issue text.