Support offloading data pre-processing to auxiliary devices

🚀 The feature, motivation and pitch

Occasionally one might find that their GPU is idle due to a bottleneck on the input data pre-processing pipeline (which might include data loading/filtering/manipulation/augmentation/etc). In these cases one could improve resource utilization by offloading some of the pre-processing to auxiliary CPU devices.
I have demonstrated how to do this using gRPC in the following blog post: https://towardsdatascience.com/overcoming-ml-data-preprocessing-bottlenecks-with-grpc-ca30fdc01bee

TensorFlow has built in (experimental) support for this feature (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service) that enables offloading in a few simple steps.

The request here is to include PyTorch APIs for offloading data pre-processing in a manner that would be simple and straight forward to the user... Similar to the TensorFlow APIs (though preferably without any limitations on pre-processing workload) .

Alternatives

No response

Additional context

No response

cc @SsnL @VitalyFedyunin @ejguan @NivekT

No repository files, tests, or entry points are named. Start by reviewing the linked gRPC example and TensorFlow data service API, then inspect the repository to determine where a comparable PyTorch API would belong; done would mean a defined, user-facing way to offload preprocessing to auxiliary devices.