llama.py with fp8 is broken (inference produces garbage results)

Hi!

II have a finetuned Llama2 and followed the example/llama.py. When I build the model in fp16, it works just fine, and produces sane results. When we use either the `--fp8` or `--fp8-cache`, the results are garbage (the same chinese character shows on repeat in the completion.

For completness, here's the script I'm calling
`python build_llama.py --max-prompt-length 1548 --max-new-tokens 500 --fp8 --fp8-cache --max-batch-size 1 /var/datamodels/llama-7b /var/data/llama-7b-optimized`

Model loading is done using
```
self.model = AutoModelForCausalLM.from_pretrained(
model_path,
use_fp8=True,
```

Generation is done using
```
generated, lengths = self.model.generate(
input_ids=torch.tensor([input_tokens]).to(self.device),
repetition_penalty=1.0,
temperature=const.OpenSourceLlmInference.temperature,
top_k=50,
top_p=0.9,
pad_token_id=self.tokenizer.pad_token_id,
eos_token_id=self.tokenizer.eos_token_id,
max_new_tokens=max_gen_tokens
)
```

My machine is an H100, so the architecture is supported. I'm using the provided docker image. Cuda and version:

NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.2

Thanks a lot for an awesome contribution, I hope we can figure this out as low latency is mission-critical for us.