[QUESTION]Are there more efficient implementations for sliding window in Megatron?

I'm working on long-text pre-training with context parallelism (CP) using Megatron. Are there more efficient implementations for sliding window training in this scenario? Currently, we're using the all_gather strategy, but the forward pass is extremely slow.it has even led to timeout issues. Could this be because the single-layer communication volume of all_gather is CP times that of full attention, leading to excessive communication overhead?

Environment details:
Megatron v0.12.0
Transformer Engine v1.12
CUDA 12.3
GPU: H100