TypeError: zeros(): argument 'size' must be tuple of ints, but found element of type float at pos 2

If you do not know the root cause of the problem, please post according to this template:

## Instructions To Reproduce the Issue:

Check https://stackoverflow.com/help/minimal-reproducible-example for how to ask good questions.
Simplify the steps to reproduce the issue using suggestions from the above link, and provide them below:

1. Full runnable code or full changes you made:
```
No changes made
```
2. What exact command you run:

python3 train_net.py --eval-only --json_annotation_train "/train.json" --json_annotation_val "test0.json" --image_path_val "/train_images" --image_path_train "test_images" --config-file "config.yaml" --num-gpus 4 MODEL.WEIGHTS "model_final.pth" OUTPUT_DIR "model_evaluation" MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE 4 SOLVER.CHECKPOINT_PERIOD 100 SOLVER.MAX_ITER 10 SOLVER.IMS_PER_BATCH 4

## Expected behavior:
Receive Inference Evaluation of the Model

##ISSUE FACING

During the inference of the model, I am receiving the following error. I tried evaluation for less number of images like 1,2 and the inference seem to work

ERROR:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
[09/01 11:08:58 d2.evaluation.evaluator]: Start inference on 412 batches
/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2895.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2895.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2895.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2895.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
[09/01 11:09:07 d2.utils.memory]: Attempting to copy inputs of to CPU due to CUDA OOM
[09/01 11:09:14 d2.evaluation.evaluator]: Inference done 1/412. Dataloading: 8.0095 s/iter. Inference: 6.8136 s/iter. Eval: 1.4886 s/iter. Total: 16.3152 s/iter. ETA=1:51:45
Traceback (most recent call last):
File "train_net.py", line 205, in
launch(
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/engine/launch.py", line 67, in launch
mp.spawn(
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 240, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
while not context.join():
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/engine/launch.py", line 126, in _distributed_worker
main_func(*args)
File "/home/ubuntu/git/ds-nlp-services/ocr/smartsearch/layout_model/LayoutModelTrain/train_net.py", line 150, in main
res = Trainer.test(cfg, model)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 617, in test
results_i = inference_on_dataset(model, data_loader, evaluator)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/evaluation/evaluator.py", line 158, in inference_on_dataset
outputs = model(inputs)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward
return self.inference(batched_inputs)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 220, in inference
return GeneralizedRCNN._postprocess(results, batched_inputs, images.image_sizes)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 248, in _postprocess
r = detector_postprocess(results_per_image, height, width)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/modeling/postprocessing.py", line 66, in detector_postprocess
results.pred_masks = roi_masks.to_bitmasks(
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/structures/masks.py", line 533, in to_bitmasks
bitmasks = paste_func(self.tensor, boxes.tensor, (height, width), threshold=threshold)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/utils/memory.py", line 70, in wrapped
return func(*args, **kwargs)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch/jit/_trace.py", line 1127, in wrapper
return fn(*args, **kwargs)
File "/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2/layers/mask_ops.py", line 129, in paste_masks_in_image
img_masks = torch.zeros(
TypeError: zeros(): argument 'size' must be tuple of ints, but found element of type float at pos 2
/usr/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 40 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d ' )

## Environment:

---------------------- -----------------------------------------------------------------------------------------
sys.platform linux
Python 3.8.13 (default, Apr 19 2022, 00:53:22) [GCC 7.5.0]
numpy 1.20.3
detectron2 0.6 @/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/detectron2
Compiler GCC 7.5
CUDA compiler CUDA 10.0
detectron2 arch flags 7.0
DETECTRON2_ENV_MODULE
PyTorch 1.12.1+cu102 @/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torch
PyTorch debug build False
GPU available Yes
GPU 0,1,2,3 Tesla T4 (arch=7.5)
Driver version 450.80.02
CUDA_HOME /usr/local/cuda
Pillow 9.2.0
torchvision 0.13.1+cu102 @/home/ubuntu/venv-collections/ocr38/lib/python3.8/site-packages/torchvision
torchvision arch flags 3.5, 5.0, 6.0, 7.0, 7.5
fvcore 0.1.5.post20210924
iopath 0.1.9
cv2 4.5.3
---------------------- -----------------------------------------------------------------------------------------
PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 10.2
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70
- CuDNN 7.6.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

Testing NCCL connectivity ... this should not hang.
NCCL succeeded.

If your issue looks like an installation issue / environment issue,
please first check common issues in https://detectron2.readthedocs.io/tutorials/install.html#common-installation-issues

Start with detectron2/layers/mask_ops.py at paste_masks_in_image, reached through structures/masks.py and modeling/postprocessing.py. Run the provided train_net.py evaluation command and inspect the image size and mask-processing path around the failing torch.zeros call. Done means the reported evaluation completes without the float-size TypeError, with an appropriate regression test or reproduction if the cause can be isolated.

Written by the indexing model from the issue text.