[BUG]After using the code that supports llama inference, the result of the inference is different from the original one

When I asked "Who is founder of goolge.com?", the result of llama13B answered as shown in the figure below:

“tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro tro accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accur accu....”

It was run on two V100s, and the configuration was as follows

model = deepspeed.init_inference(
model=model,
mp_size=2,
dtype=torch.float16,
replace_method="auto",
replace_with_kernel_inject=True,
)

Start by reproducing the llama13B inference on two V100s with the shown deepspeed.init_inference configuration, including mp_size=2, float16, and kernel injection. Compare the output with the original inference path and investigate the model-parallel and kernel-injection paths; done requires identifying the cause of the repeated-token output and verifying corrected inference behavior.