Update the document to keep non-replicable DataPipe in the main process for MPRS

🚀 The feature

Highlight the fact that the MPRS attaches non-replicable datapipe branches at the end of it in the documentation.
Also mention the currently undocumented / obscure is_replicable() interface for IterDataPipe and MapDataPipe.

Old Issue

I would like to make the use of MultiProcessingReadingService as opaque as possible. In particular, it should not affect the shuffling behavior. For that, I need more control and be able to manually expand the internal pipeline of the MultiProcessingReadingService AFTER worker batches are received in a round-robin fashion such that I could e.g. add an additonional shuffle buffer. It would be good to have an extension point for this. Potentially, the Adapter protocol could be expanded for this.

Also, on a side note: There should possibly be an Adapter that divides the shuffle buffer size of shuffle operations, so that the overall memory consumption does not depend on the amount of IO workers. The existing Shuffle adapter could be expanded for this. While I am aware that the documentation recommends placing shuffling operations as early as possibly where they still might act on a logical level, there can be situations where this is simply not possible and shuffling has to be done at the end.

Motivation, pitch

I'm working with horovod which spawns one MPI process per GPU on each node. Logical sharding of the dataset takes place in this MPI setting and usage of the MultiProcessingReadingService (per process/gpu/shard) should be seen as an implementation detail. As such, I don't want it to affect the shuffling behavior within a shard, i.e. there should be no difference in randomness between training with 1 IO worker or training with 8 IO workers per GPU.

In my particular case, shuffling also needs to be performed right at the end of the pipeline.

Locate the documentation for MultiProcessingReadingService and IterDataPipe/MapDataPipe interfaces, then review how the current pipeline behavior is described. Done means the documentation explains that non-replicable DataPipe branches remain in the main process and documents the is_replicable() interface; check the rendered documentation for both additions.