pretrained model with lazyconfig cannot inference on cpu even though the device is set to cpu

## Instructions To Reproduce the 🐛 Bug:
1. Full runnable code or full changes you made:
```
# git rev-parse HEAD
# 38af375052d3ae7331141bc1a22cfa2713b02987

import torch
from detectron2 import model_zoo
import detectron2.config
cfg = model_zoo.get_config("new_baselines/mask_rcnn_R_50_FPN_400ep_LSJ.py", trained=True)
cfg.model.update(input_format='RGB')
cfg.model.roi_heads.box_predictor.update(test_score_thresh=0.1) # set threshold for this model
model = detectron2.config.instantiate(cfg.model)
# model =model.to("cuda") # works if uncommented
model.eval()
print(f"model.device={model.device}, model.training={model.training}")
input_size = 640
image = torch.rand(3, input_size, input_size, dtype=torch.float32)
# image = image.to("cuda") # works if uncommented
outputs = model([{"image": image}])
```
2. What exact command you run:
3. __Full logs__ or other relevant observations:
```
ValueError Traceback (most recent call last)
Cell In[3], line 12
10 input_size = 640
11 image = torch.rand(3, input_size, input_size, dtype=torch.float32).to("cpu")
---> 12 outputs = model([{"image": image}])

File venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1194, in Module._call_impl(self, *input, **kwargs)
1190 # If we don't have any hooks, we want to skip the rest of the logic in
1191 # this function, and just call forward.
1192 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1193 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1194 return forward_call(*input, **kwargs)
1195 # Do not call functions when jit is used
1196 full_backward_hooks, non_full_backward_hooks = [], []

File detectron2/detectron2/modeling/meta_arch/rcnn.py:150, in GeneralizedRCNN.forward(self, batched_inputs)
127 """
128 Args:
129 batched_inputs: a list, batched outputs of :class:`DatasetMapper` .
(...)
147 "pred_boxes", "pred_classes", "scores", "pred_masks", "pred_keypoints"
148 """
149 if not self.training:
--> 150 return self.inference(batched_inputs)
...
--> 686 raise ValueError("SyncBatchNorm expected input tensor to be on GPU")
688 self._check_input_dim(input)
689 self._check_non_zero_input_channels(input)

ValueError: SyncBatchNorm expected input tensor to be on GPU
```
4. please simplify the steps as much as possible so they do not require additional resources to
run, such as a private dataset.

## Expected behavior:

If there are no obvious error in "full logs" provided above,
please tell us the expected behavior.

It should not require gpu to run the model if the device is set to cpu.

Or at least by default set the model device to GPU. I actually cannot find anyway to change the model device with lazyconfig in the document. I have to dig into the code to find out I need `model=model.to("cuda")`

However, we would also like to demonstrate the mode without GPU.

Does this means the pretrained lazyconfig model can only stick on device has been trained on? This will be very annoying.

## Environment:

Provide your environment information using the following command:
```
---------------------- ------------------------------------------------------------------------------------------------------
sys.platform linux
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
numpy 1.24.2
detectron2 0.6 @~/detectron2/detectron2
Compiler GCC 11.3
CUDA compiler CUDA 11.7
detectron2 arch flags 7.5
DETECTRON2_ENV_MODULE
PyTorch 1.13.1+cu117 @~/venv/lib/python3.10/site-packages/torch
PyTorch debug build False
GPU available Yes
GPU 0 NVIDIA GeForce RTX 2070 (arch=7.5)
Driver version 525.85.05
CUDA_HOME ~/cudas/cuda-11.7b
Pillow 9.4.0
torchvision 0.14.1+cu117 @~/venv/lib/python3.10/site-packages/torchvision
torchvision arch flags 3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore 0.1.5.post20221221
iopath 0.1.9
cv2 Not found
---------------------- ------------------------------------------------------------------------------------------------------
PyTorch built with:
- GCC 9.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.7
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
- CuDNN 8.5
- Magma 2.6.1
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
```

If your issue looks like an installation issue / environment issue,
please first try to solve it yourself with the instructions in
https://detectron2.readthedocs.io/tutorials/install.html#common-installation-issues

Reproduce the runnable example with the lazyconfig model and follow the traceback into detectron2/modeling/meta_arch/rcnn.py at GeneralizedRCNN.forward and PyTorch's SyncBatchNorm check. Compare the CPU and CUDA paths and inspect how the instantiated model's device and normalization layers are configured. Done means the documented CPU path completes inference without requiring a GPU, with tests or documentation covering the supported behavior.

Written by the indexing model from the issue text.