RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

使用 decoupleQ 量化了一个 Llama-2-7b-hf 模型，使用 run_inference_llama.sh 进行推理时，报错如下：
```
Traceback (most recent call last):
File "/mnt/afs/quantization/test/decoupleQ/llama.py", line 476, in
model_output = model.generate(input_token_ids_tensor, max_length=40)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/miniconda3/envs/decoupleQ/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/miniconda3/envs/decoupleQ/lib/python3.11/site-packages/transformers/generation/utils.py", line 1758, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/usr/local/lib/miniconda3/envs/decoupleQ/lib/python3.11/site-packages/transformers/generation/utils.py", line 2437, in _sample
next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```
通过添加do_sample=False
```
model_output = model.generate(input_token_ids_tensor, max_length=40, do_sample=False)
```
推理可以继续运行，但是输出直接乱掉：
```
out_text: [' who are you?']
inference speed: e2e 1007.3947906494141 ms, pertoken 25.18486976623535 ms
```
请问下，你们有遇到这种情况吗？是否有对应的解决办法？
谢谢！