[QUESTION][dev] Inconsistent Loss Values in Multiple Runs of DeepSeek V4 Demo with Identical Configurations

## Question
DeepSeek V4 Demo(only 4 layer + 1 mtp) presents non-reproducible loss results. Multiple independent runs with all experimental configurations fixed produce inconsistent loss values. This random fluctuation breaks experimental stability and reproducibility, interfering with model validation, performance comparison and ablation analysis.
## Reproduce
environment:
- 8*H800
- python 3.12
- pytorch 2.7.1+cu128
- transformer_engine 2.8.0+40c69e75
- fast-hadamard-transform 1.1.0
- flash attn 3.0.0
branch: dev
commit: 630956b35
shell:
[mcore_run.sh](https://github.com/user-attachments/files/28621736/mcore_run.sh)
logs:
[7_0_stdout.log](https://github.com/user-attachments/files/28621671/7_0_stdout.log)
[7_1_stdout.log](https://github.com/user-attachments/files/28621670/7_1_stdout.log)
[7_2_stdout.log](https://github.com/user-attachments/files/28621669/7_2_stdout.log)

loss when 10 iters:
lm loss: 7.177544E+00 | load_balancing_loss: 0.000000E+00 | mtp_1 loss: 6.928595E+00
lm loss: 7.177759E+00 | load_balancing_loss: 0.000000E+00 | mtp_1 loss: 6.928655E+00
lm loss: 7.178733E+00 | load_balancing_loss: 0.000000E+00 | mtp_1 loss: 6.924607E+00

@hxbai @Victarry Please check whether you can reproduce it. If not, please leave a comment.