`FullSync` gives distributed not initialized error for old `DataLoader`

🐛 Describe the bug

Hi, here we are training in webdataset format with torchdata. Everything else works fine without fullsync, just the training hangs at the last iteration. So I tried to apply fullsync to my datapipe, then it would give the following error:

RuntimeError: Caught RuntimeError in DataLoader worker process 0.
  File "/root/miniconda3/lib/python3.8/site-packages/torch/utils/data/datapipes/_hook_iterator.py", line 173, in wrap_generator
    response = gen.send(None)
  File "/root/miniconda3/lib/python3.8/site-packages/torchdata/datapipes/iter/util/prefetch.py", line 173, in __iter__
    raise RuntimeError("Torch Distributed is required to be initialized")
RuntimeError: Torch Distributed is required to be initialized
This exception is thrown by __iter__ of FullSyncIterDataPipe()

My training scirpt:

def build_torchdata_loader(rootdir):
    import torch.distributed as ddist
    # both true
    print("dist: ", ddist.is_available(), ddist.is_initialized())
    assert ddist.is_initialized()

    tar_files = list(glob.glob('{}/*.tar'.format(rootdir)))

    train_dp = dp.iter.FileLister(tar_files).shuffle().sharding_filter()
    
    # the last iter hangs
    # train_dp = dp.iter.FileOpener(train_dp, mode="rb").load_from_tar().webdataset()
    # dist not init error
    train_dp = dp.iter.FileOpener(train_dp, mode="rb").load_from_tar().webdataset().fullsync()

    train_dp = train_dp.shuffle(buffer_size=128)
    train_dp = train_dp.map(postprocess_func)

    train_loader = DataLoader(train_dp, batch_size=128, num_workers=8, pin_memory=True)
    return train_loader

Also, I'm not sure if this is the correct way to use torchdata with dataLoader and ddp. I only managed to find the colab tutorial with dataloader2 ddp. I'm wondering if it is possible to also provide one for the original dataloader.

Many thanks in advance!

Versions

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==1.13.1
[pip3] torchdata==0.5.1
[pip3] torchvision==0.14.1

Start with the build_torchdata_loader example and the FullSyncIterDataPipe error from torchdata 0.5.1, then compare how the original DataLoader and distributed initialization are used with num_workers=8. Check whether the reported failure and last-iteration hang can be reproduced; done means the supported usage or required documentation is clear for this setup.