getting random tokens after finetuning

Nobody has claimed this yet.

Hi Everyone,
I followed the steps for finetuning the llama model from https://github.com/Lightning-AI/lit-llama and did the fine-tuning.
when getting the results from the model on validation dataset while using the model to generate different inputs with same model object it is generating some random tokens can anyone know how can i control the random tokens and get the refined result?
has anyone tried for batch generation for different inputs if yes how can we use the lit-llama functionality for batch prediction ?

Start with the linked lit-llama fine-tuning instructions and the validation-generation path described in the issue. Reproduce generation with the same model object and inputs, then investigate whether the token variation is expected or indicates a reproducibility problem. Also determine whether batch prediction is supported; done means the behavior and any required usage are clearly established.

Written by the indexing model from the issue text.