os.environ['CUDA_VISIBLE_DEVICES'] usage error

Nobody has claimed this yet.

In your initalization.py module, you have the following code:

if args.gpu_device_id is not None:
os.environ['CUDA_VISIBLE_DEVICES'] = str(args.gpu_device_id)
print("Set device to:", args.gpu_device_id)

In Windows, this usage is NOT doing what you apparently think it is. A common misconception is that setting this variable to the desired device id will make only THAT device available.

In fact, it DOES NOT do that. For instance, if I have 2 GPUs, 0 and 1, and I want to use GPU ID 1, your code will set os.environ['CUDA_VISIBLE_DEVICES'] = "1". What this does is set the number of devices available to 1, and makes the only device available GPU 0. This is obviously NOT what is desired.

The fix (for cuda devices) is to either NOT set the os.environ['CUDA_VISIBLE_DEVICES'] variable at all, or special case it the devices are "cuda", and instead, change your code in memory_management.py to:

def get_torch_device():
global directml_enabled
global cpu_state
if directml_enabled:
global directml_device
return directml_device
if cpu_state == CPUState.MPS:
return torch.device("mps")
if cpu_state == CPUState.CPU:
return torch.device("cpu")
else:
if is_intel_xpu():
return torch.device("xpu", torch.xpu.current_device())
else:
if not args.gpu_device_id is None:
return torch.device(int(args.gpu_device_id))
else:
return torch.device(torch.cuda.current_device())

Start by tracing GPU selection from initialization.py into memory_management.py, focusing on how gpu_device_id and CUDA device state are handled. Exercise the Windows CUDA path and the existing CPU, MPS, Intel XPU, and DirectML branches; done means the requested GPU is selected without incorrect CUDA visibility behavior and other device modes remain intact.

Written by the indexing model from the issue text.