[QUESTION] the CUDA Stream Management for CUDA Graph

**Your question**
Ask a clear and concise question about Megatron-LM.
I'm currently exploring the CUDA Graph functionality in Megatron-LM and have a question regarding the stream management logic.I've noticed a specific pattern in how CUDA streams are handled for graph capturing:
1. Initialization with a non-default stream: During the initialization phase , a new, non-default stream is created and set for the current device using torch.cuda.set_stream(torch.cuda.Stream()). The stated purpose is "Set to non-default stream for cudagraph capturing".
2. Switching back to the default stream before capture: However, within the cuda_graph_capture function, right before the call to _make_graphed_callables where the actual capture happens, the stream is switched back to the default stream.

Relevant Code : https://github.com/NVIDIA/Megatron-LM/commit/a606486ecefa46eafafe771cf8177195099f11e7

I am trying to understand the rationale behind this design.Why is it necessary to first set a non-default stream during initialization, only to switch back to the default stream immediately before the capture process begins?

Thank you for your time and for your amazing work on Megatron-LM.