Resnet DDP Warning: grad strides do not match bucket view strides

Description

On imagenet_resnet workload, I encounter the following warning when running with DDP and pytorch framework.

/u/najroldi/miniconda3/envs/alpe/lib/python3.8/site-packages/torch/autograd/init.py:251: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [512, 2048, 1, 1], strides() = [2048, 1, 2048, 2048]

Is this a known problem? Is there a known fix?

Related to this, I have noticed that the prize_qualification_baseline logs start with python3 submission_runner.py. For self reporting results, and for the final scoring, should we use DDP?

Steps to Reproduce

torchrun --redirects 1:0,2:0,3:0,4:0,5:0,6:0,7:0 --standalone --nnodes=1 --nproc_per_node=8 submisison_runner.py --framework=pytorch --workload=imagenet_resnet ...

Start by running the reported torchrun command with submission_runner.py, the pytorch framework, and the imagenet_resnet workload, then inspect the DDP warning and its gradient stride details. Determine whether the warning has a known resolution and clarify whether DDP is expected for self-reported results and final scoring; done means the issue has an evidence-backed answer or documented guidance.