Abnormally slow both single-gpu & DDP training, what is the problem here?

Nobody has claimed this yet.

### Bug description

I just adapt my training into lightning framework for convenient ddp model training. But I got almost 10 times slower than my previous manually torch ddp training, the speed is shown below. I have no idea what is wrong here, could anyone help me figure out what may caus this problem and how to fix it?

I have set:
```
pl.Trainer(
accelerator='gpu',
devices=#GPUS,
strategy='ddp',
sync_batchnorm=True,
deterministic=True,
gradient_clip_val=$CLIP_VALUE
)

```

### What version are you seeing the problem on?

v2.3

### How to reproduce the bug

```python

```

### Error messages and logs

```
# Error messages and logs here please
```

### Environment

Current environment

```
#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
```

### More info

_No response_

cc @justusschock @lantiga

Start with the Trainer configuration in the issue, especially accelerator='gpu', strategy='ddp', sync_batchnorm, deterministic, and gradient clipping. The issue provides no reproduction code, logs, environment details, files, or tests, so first collect a runnable comparison of the Lightning and manual PyTorch DDP cases. Done means identifying the cause of the slowdown and documenting or validating a fix against the reported single-GPU and DDP behavior.

Written by the indexing model from the issue text.