Question: Performance of multi-device prefetching

Hello,

I have a quick question with regards to the implementation of the multi-device put and prefetching iterator. From looking at the code, it seems that there is quite a lot of unnecessary computation that occurs that can inhibit multi device learning actually speeding up computation. From my understanding, what happens is that for each device, the iterator calls next so in the case of 8 devices 8 next calls are done sequentially. Would it not be much faster to simply make a single call and split it. In some experiments I have ran, the speed of the multi-device put is massively slower and upon inspecting the profiler, it's all due to the synchronous calls to next. A solution to this is using prefetching and raising the prefetching thread count but it's still slower than performing the same code on a single device. I was just wondering if there is any specific reason you have opted for sequential next calls instead of a single next call and then data split?