Does anyone have issue with GPU parallelization for X3D using Pytorch from source?

In order to use the Conv3D pull request, I installed the latest Pytorch, torchvision, and detectron2 from source. I used to have no problem with multi-GPU parallelization with torch 1.7.1, but this time I notice that although the model can run without issue, but it seems like every GPU is running the program as they would if there is only 1 GPU.

For example, I have 4 GPUs, then I will see:
`INFO:train_net:Start epoch: 1 ` 4 times when using 4 GPUs.
Also my dataloader also print 4 times about loading train set, val set.

If I switch back to torch 1.7.1, then it will loads one copy of data and actually distribute it to 4 GPUs given batch_size/4. What could be the problem?