Attention across batch in forward train

Hi,

In the training code released, the videos are first flattened along the number of frames and batch dimension before being passed to the image encoder
https://github.com/facebookresearch/sam2/blob/2b90b9f5ceec907a1c18123530e92e794ad901a4/training/model/sam2.py#L110C13-L110C68

However, this means that self-attention will be applied across tokens from different videos, is this expected/ok behavior?

Thanks.