When using mini-batch, there is a potential risk of future information leakage

When I'm training a MLP model with mini-batched input, the result is more better than the pytorch's version. I checked my input, I found my input samples are time continuous data and I forgot make a global shuffle. So the model can see future information in a mini-batch. I think the main reason is the low-level implementation of AffineTransform function with mini-batch. One solution is to make a global shuffle before making mini-batch, another solution maybe is to optimize the implementation of mini-batched AffineTransform. Thanks for paying attention to this issue.

Start by reproducing the mini-batch behavior with time-continuous input and compare it with the PyTorch result. Then inspect the low-level AffineTransform implementation for mini-batches, as identified in the report, and determine whether the issue is implementation leakage or missing global shuffling. Done requires a confirmed cause and a decided correction path.

Written by the indexing model from the issue text.