Shuffle order is the same across runs when using strategy='ddp'

Bug description

The batches and their order are the same across different executions of the script when using strategy='ddp' and dataloader with shuffle=True

What version are you seeing the problem on?

v2.2

How to reproduce the bug

Say you have train.py that prints the current input on each training iteration and has shuffling enabled in the
dataloader:

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn.functional as F
import lightning.pytorch as pl

class SomeLightningModule(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.p1 = torch.nn.Parameter(torch.tensor(0.0))
        self.p2 = torch.nn.Parameter(torch.tensor(0.0))
    def training_step(self, batch):
        x, y = batch
        print(x.item())
        return F.mse_loss(x * self.p1 + self.p2, y)
    def configure_optimizers(self):
        optimizer = torch.optim.Adam(
            self.parameters(),
        )

        return {
            "optimizer": optimizer,
        }

lightning_module = SomeLightningModule()

trainer = pl.Trainer(
    strategy='ddp',
    max_epochs=1,
)

train_dataset = TensorDataset(torch.arange(5).float(), torch.arange(5).float())
train_loader = DataLoader(train_dataset, shuffle=True)

trainer.fit(lightning_module, train_dataloaders=train_loader)

When strategy='ddp', the script will print the same numbers across different runs:

$ python3 train.py
4.0
0.0
1.0
3.0
2.0
$ python3 train.py
4.0
0.0
1.0
3.0
2.0

Such behavior can be unwanted, as people might want to try different orders of batches (e.g. to construct ensembles or get the average performance)

Error messages and logs

# Error messages and logs here please

Environment

More info

No response

cc @justusschock @lantiga

Beginne damit, das Problem mit dem bereitgestellten train.py-Beispiel zu reproduzieren, unter Verwendung von DataLoader(shuffle=True) und strategy='ddp'. Verfolge, wie DDP das Shuffling und Seeding des Dataloaders behandelt; abgeschlossen ist die Aufgabe, wenn separate Ausführungen unterschiedliche Batch-Reihenfolgen erzeugen, ohne das Verhalten des verteilten Trainings zu beeinträchtigen.