RuntimeError: CUDA error: invalid device ordinal

I'm trying to train MViT from a pretrained model using

`
!python tools/run_net.py \
--cfg configs/Kinetics/MVIT_B_16x4_CONV.yaml \
DATA.PATH_TO_DATA_DIR path/to/dataset/`

When I run this code cell, I receive the following errors:

````
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_functional_video.py:5: UserWarning: The _functional_video module is deprecated. Please use the functional module instead.
warnings.warn(
/home/slubuntu/anaconda3/lib/python3.8/site-packages/torchvision/transforms/_transforms_video.py:25: UserWarning: The _transforms_video module is deprecated. Please use the transforms module instead.
warnings.warn(
Traceback (most recent call last):
File "tools/run_net.py", line 44, in
main()
File "tools/run_net.py", line 25, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File "/home/slubuntu/Documents/train/slowfast/slowfast/utils/misc.py", line 296, in launch_job
torch.multiprocessing.spawn(
File "/home/slubuntu/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 230, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/slubuntu/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 188, in start_processes
while not context.join():
File "/home/slubuntu/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 150, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 6 terminated with the following error:
Traceback (most recent call last):
File "/home/slubuntu/anaconda3/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 59, in _wrap
fn(i, *args)
File "/home/slubuntu/Documents/train/slowfast/slowfast/utils/multiprocessing.py", line 59, in run
torch.cuda.set_device(local_rank)
File "/home/slubuntu/anaconda3/lib/python3.8/site-packages/torch/cuda/__init__.py", line 264, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.`
```

How can I fix this?