Investigate the difference in speedup between Llama3-8b and Mistral-7B-v0.1

Nobody has claimed this yet.

## 🚀 Feature

Make Thunder + Mistral-7B-v0.1 as fast as Thunder + Llama3-8b (comparing to Eager mode).

### Motivation
Below are data for:
* Llama3-8b:


* Mistral-7B-v0.1:


* Plot comparing them:

The main difference is that Mistral-7B-v0.1 fits into 1 GPU, but Llama3-8b not, so for Llama3-8b we use distributed training.
* Llama-3-8b config:
```
# https://huggingface.co/meta-llama/Meta-Llama-3-8B/blob/main/config.json
dict(
name="Llama-3-8B{}",
hf_config=dict(org="meta-llama", name="Meta-Llama-3-8B{}"),
block_size=8192,
vocab_size=128000,
padded_vocab_size=128256,
n_layer=32,
n_head=32,
n_query_groups=8,
rotary_percentage=1.0,
parallel_residual=False,
bias=False,
norm_class_name="RMSNorm",
mlp_class_name="LLaMAMLP",
intermediate_size=14336,
rope_base=500000
)
```
* Mistral-7B-v0.1 config:
```
dict(
name="Mistral-7B-{}v0.1",
hf_config=dict(org="mistralai", name="Mistral-7B-{}v0.1"),
padded_vocab_size=32000,
block_size=4096, # should be 32768 but sliding window attention is not implemented
n_layer=32,
n_query_groups=8,
rotary_percentage=1.0,
parallel_residual=False,
bias=False,
norm_class_name="RMSNorm",
norm_eps=1e-05,
mlp_class_name="LLaMAMLP",
intermediate_size=14336,
)

```

Start by reproducing the Llama-3-8B and Mistral-7B-v0.1 comparisons described in the issue, using their supplied model configurations and the Thunder-versus-Eager measurements. Investigate the effect of single-GPU versus distributed training and determine what change would make the speedups comparable; done means the performance gap is explained and the resulting measurements meet the stated goal.

Written by the indexing model from the issue text.