[BUG] some question about flops_profiler

Describe the bug
question:

1. when I use flops_profiler to profile TFLOPS of Megatron GPT, I get the value not correct.
2. when I use flops_profiler to train LLAMA2, I can't get any infos.

To Reproduce
Steps to reproduce the behavior:
for question 1:

1. go to the project https://github.com/microsoft/Megatron-DeepSpeed,
2. modify the shell script https://github.com/microsoft/Megatron-DeepSpeed/blob/main/examples_deepspeed/run_deepspeed_example.sh, add "flops_profiler" in ds_config, then I get the different value from which Megatron returns to me.
   for question 2:
3. go to the project https://github.com/hiyouga/LLaMA-Factory/tree/main
4. use deepspeed for training llama v2 and add "flops_profiler" in ds_config. I can't get any info about TFLOPS

Expected behavior

1. get the same TFLOPS as Megatron returns to me
2. get some info when I am to train the llama v2