PyTorch Profiler Stats Only Showing for "Records"

Nobody has claimed this yet.

### Bug description

Hello Lightning,
I am trying to use PyTorch Lightning Profiler (Lightning 1.9.0) to find bottlenecks for my script and am encountering strange outputs.
Here is an example Profiler output file (3 more tables not shown):


I notice that the tables are titled “FIT Profiler Report”, and that the stats are for “record rank: 0”.
I expected output similar to the Lightning Docs, where we’d see “Profile stats for: training_step”.
https://lightning.ai/docs/pytorch/LTS/tuning/profiler_intermediate.html?highlight=profiler

I am getting some negative time measurements too. There is another ticket that shows a similar issue to what I am encountering, but it has not been resolved.
#https://lightning.ai/forums/t/pytorch-profiler-only-reports-stats-for-records/1104

Context: We are experimenting with 3D parallelism with NeMo Megatron using Pytorch Lightning Profiler, and we want to see CPU Utilization.

Expected Behavior:
“Profile stats for: training_step“
A single table for each rank showing metrics for the training_step.

Actual Behavior:
“Profile stats for: records”
5 tables for each rank where tables are not differentiated in title. (We are using 5 node pipeline parallelism)

Could someone explain the output and why it looks so different than expected in the lightning docs: https://lightning.ai/docs/pytorch/LTS/tuning/profiler_intermediate.html?highlight=profiler
I am using Lightning 1.9.0

### What version are you seeing the problem on?

v1.9

### How to reproduce the bug

_No response_

### Error messages and logs

```
# Error messages and logs here please
```

### Environment

Current environment

```
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):
```

### More info

_No response_

cc @carmocca @nbcsm @guotuofeng

Start by reproducing the PyTorch Lightning Profiler output described for Lightning 1.9.0, comparing the reported “records” tables with the profiler documentation linked in the issue. Investigate the five-node pipeline-parallel context and negative time measurements; done means explaining or correcting the output so profiling reports the expected training_step statistics per rank.

Written by the indexing model from the issue text.