[optim][ub] pt2 values keep flip-flopping nondeterministically

TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 806d1a871ddfd2d38e1791489892009feaec8425

Affected PyTorch commit: 3316374d14f2e488b65dfb4d0a2560d8414fa19f

Affected Tests:

• timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: +482341.57958%
• timm_vision_transformer_large, RAdam, cuda, differentiable: +81.55424%
• timm_vision_transformer, AdamW, cuda, (pt2) no_foreach: -99.98224%
• timm_vision_transformer, AdamW, cuda, (pt2) foreach: -66.83298%
• compile_time, timm_vision_transformer, AdamW, cuda, (pt2) no_foreach: +1891.78438%
• compile_time, timm_vision_transformer, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +895.46175%
• compile_time, timm_vision_transformer, RMSprop, cuda, (pt2) foreach: +2183.02906%
• DALLE2_pytorch, AdamW, cuda, fused, capturable, amsgrad: +38.77857%
• resnet18, Adadelta, cuda, (pt2) no_foreach: +735169.67855%
• resnet18, AdamW, cuda, (pt2) no_foreach: +606511.29533%
• resnet18, AdamW, cuda, (pt2) foreach: +192.09611%
• compile_time, resnet18, Adadelta, cuda, (pt2) foreach: +1824.75319%
• compile_time, resnet18, Adagrad, cuda, (pt2) foreach: +2646.24568%
• compile_time, resnet18, AdamW, cuda, (pt2) foreach: +1720.93538%

Tests that were no longer run on affected commit:

Tests that were newly added on affected commit:

Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6141749176

cc @janeyx99

Start by inspecting the error logs from workflow run 6141749176 and compare the listed affected tests between commits 806d1a871ddfd2d38e1791489892009feaec8425 and 3316374d14f2e488b65dfb4d0a2560d8414fa19f. Done means identifying why the PT2 values flip nondeterministically and documenting or correcting the regression so the affected benchmark results are stable.