Add batch-wise random erasing support?

🚀 Feature

Motivation

The random erasing supported in torchvision now is implemented as image-wise. I need to use it after data loader collecting batches. But the current design cannot be used for a large batch because it is too slow to preprocess on CPU.

I use a for loop to process per-image in a batch, as shown below, but it is too slow:

for batch_of_imgs, labels in my_data_loader:
    # I need apply random erasing here, not in transforms of data loader
    # This for-loop is too slow!!!
    for i, img in enumerate(batch_of_imgs):
        batch_of_imgs[i] = RandomErasing(img)
    # train model as usual
    preds = model(batch_of_imgs, labels)

Pitch

If a batch-wise random erasing operation that could accelerate such preprocessing is available, it would be very helpful.

Alternatives

I've tried 2 alternatives:

1. Use multi-threading. This does not help due to python GIL.
2. Use multi-processing. This saves around 50% preprocessing time per-iteration at the beginning of training but gradually becomes as slow as single-processing one. It provides around <10% improvements from the whole training procedure but requires more resources for processing management, and tricky hacks to modify training code.

Additional context

The erasing procedure should be image-wise, not batch-wise. In other words, the erased region cannot be shared among images of same batch.

cc @SsnL @VitalyFedyunin @ejguan @fmassa @vfdev-5

Start by locating torchvision's existing RandomErasing implementation and its related tests. Define a batch-capable interface that keeps erased regions independent per image, then benchmark it against the reported per-image loop and add coverage for the batch behavior.